Jul 10 20:37:43.405839 ip-10-0-141-228 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 20:37:43.750770 ip-10-0-141-228 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 20:37:43.750770 ip-10-0-141-228 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 20:37:43.750770 ip-10-0-141-228 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 20:37:43.750770 ip-10-0-141-228 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 20:37:43.750770 ip-10-0-141-228 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 20:37:43.751507 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.751423    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 20:37:43.757682 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757659    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:37:43.757682 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757678    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:37:43.757682 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757681    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:37:43.757682 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757684    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:37:43.757682 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757687    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757690    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757694    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757698    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757700    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757703    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757707    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757709    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757712    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757714    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757717    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757719    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757722    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757726    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757731    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757733    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757737    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757741    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757746    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:37:43.757835 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757749    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757752    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757760    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757763    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757765    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757768    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757770    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757772    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757775    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757777    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757780    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757783    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757785    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757787    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757790    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757793    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757796    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757798    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757801    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:37:43.758317 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757803    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757806    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757808    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757810    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757813    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757817    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757821    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757858    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757862    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757865    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757867    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757870    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757872    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757875    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757877    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757880    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757882    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757885    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757887    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757889    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:37:43.758799 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757894    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757896    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757898    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757901    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757904    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757906    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757909    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757911    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757915    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757917    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757920    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757922    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757925    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757927    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757930    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757932    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757934    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757937    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757939    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757945    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757948    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:37:43.759278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757952    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757955    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757957    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757960    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757962    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757965    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757967    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757969    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757972    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757974    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757976    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757979    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757981    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757984    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757986    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.757990    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758753    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758761    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758765    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:37:43.759802 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758769    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758773    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758776    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758779    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758781    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758784    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758786    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758789    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758791    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758794    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758796    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758799    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758802    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758805    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758808    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758811    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758813    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758816    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758818    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758821    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:37:43.760265 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758823    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758826    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758828    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758830    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758833    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758836    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758838    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758841    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758843    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758846    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758849    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758851    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758860    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758863    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758865    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758868    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758870    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758873    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758875    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:37:43.760807 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758878    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758880    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758883    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758885    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758888    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758890    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758893    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758896    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758898    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758901    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758903    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758906    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758909    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758911    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758914    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758916    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758919    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758924    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758927    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758930    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:37:43.761272 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758933    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758936    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758939    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758941    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758944    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758949    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758953    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758955    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758958    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758960    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758963    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758965    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758968    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758970    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758973    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758975    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758978    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758980    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758983    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758986    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:37:43.761775 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758988    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758991    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758993    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.758996    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759000    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759003    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759006    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759009    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759012    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759015    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759018    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759020    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759023    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759026    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759028    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759030    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759033    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759400    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759405    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:37:43.762258 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759407    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759410    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759413    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759415    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759418    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759420    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759422    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759425    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759427    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759429    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759432    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759434    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759439    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759442    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759445    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759447    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759449    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759452    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759455    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759457    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:37:43.762837 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759459    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759462    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759464    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759467    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759469    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759472    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759474    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759477    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759479    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759482    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759485    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759488    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759492    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759494    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759497    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759499    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759502    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759504    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759506    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:37:43.763331 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759509    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759512    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759517    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759520    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759523    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759526    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759529    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759532    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759535    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759537    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759540    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759542    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759548    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759551    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759575    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759581    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759586    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759590    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759594    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:37:43.763804 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759596    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759599    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759602    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759604    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759607    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759610    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759613    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759616    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759618    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759621    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759623    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759625    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759628    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759630    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759632    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759635    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759638    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759640    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759644    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759646    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:37:43.764278 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759649    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759652    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759654    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759657    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759660    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759664    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759666    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759670    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759673    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759675    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759678    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759680    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759683    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759685    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759688    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759690    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759693    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759695    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.759697    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759774    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 20:37:43.764789 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759781    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759787    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759791    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759795    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759799    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759803    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759807    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759810    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759813    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759817    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759821    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759824    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759827    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759830    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759833    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759835    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759838    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759844    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759847    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759851    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759861    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759866    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759870    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 20:37:43.765263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759873    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759876    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759880    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759883    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759885    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759888    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759891    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759894    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759898    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759901    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759903    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759906    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759909    2565 flags.go:64] FLAG: --enable-server="true"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759912    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759916    2565 flags.go:64] FLAG: --event-burst="100"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759919    2565 flags.go:64] FLAG: --event-qps="50"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759921    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759924    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759927    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759931    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759934    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759936    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759939    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759943    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759946    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 20:37:43.765834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759948    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759952    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759955    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759958    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759960    2565 flags.go:64] FLAG: --feature-gates=""
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759964    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759967    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759970    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759973    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759976    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759979    2565 flags.go:64] FLAG: --help="false"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759982    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-141-228.ec2.internal"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759985    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759988    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759991    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759994    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.759997    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760000    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760002    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760005    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760008    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760011    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760014    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760017    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 10 20:37:43.766479 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760020    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760023    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760026    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760028    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760032    2565 flags.go:64] FLAG: --lock-file=""
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760034    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760037    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760040    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760045    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760049    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760052    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760054    2565 flags.go:64] FLAG: --logging-format="text"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760057    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760060    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760063    2565 flags.go:64] FLAG: --manifest-url=""
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760066    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760070    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760073    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760077    2565 flags.go:64] FLAG: --max-pods="110"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760080    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760083    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760086    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760089    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760091    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760094    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 20:37:43.767076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760097    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760104    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760107    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760110    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760114    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760116    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760122    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760124    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760127    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760130    2565 flags.go:64] FLAG: --port="10250"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760133    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760136    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03a14a19cfcbdc2c1"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760139    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760142    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760145    2565 flags.go:64] FLAG: --register-node="true"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760148    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760153    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760156    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760159    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760161    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760165    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760168    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760171    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760173    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760176    2565 flags.go:64] FLAG: --runonce="false"
Jul 10 20:37:43.767688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760179    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760182    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760185    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760187    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760190    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760193    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760196    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760199    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760202    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760205    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760207    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760210    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760213    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760216    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760218    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760223    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760226    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760229    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760233    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760237    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760240    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760243    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760246    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760251    2565 flags.go:64] FLAG: --v="2"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760255    2565 flags.go:64] FLAG: --version="false"
Jul 10 20:37:43.768311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760259    2565 flags.go:64] FLAG: --vmodule=""
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760263    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.760266    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760381    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760384    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760387    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760390    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760393    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760396    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760398    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760401    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760404    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760407    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760410    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760412    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760415    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760417    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760420    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760422    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760425    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:37:43.768993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760428    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760430    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760432    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760435    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760437    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760439    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760443    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760446    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760448    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760451    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760455    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760458    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760460    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760463    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760465    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760467    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760470    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760472    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760474    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760477    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:37:43.769486 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760479    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760481    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760484    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760486    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760488    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760491    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760493    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760495    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760498    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760500    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760503    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760505    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760508    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760511    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760514    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760517    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760519    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760521    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760525    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:37:43.770099 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760527    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760530    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760532    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760536    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760538    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760541    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760544    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760546    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760548    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760551    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760566    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760570    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760572    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760575    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760578    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760580    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760582    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760585    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760588    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760591    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:37:43.770667 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760593    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760596    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760599    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760601    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760604    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760606    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760608    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760611    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760613    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760616    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760618    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760624    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760627    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760630    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760633    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760637    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760639    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760642    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760645    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:37:43.771354 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760649    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760652    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760655    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.760658    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.761272    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.767913    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.767929    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.767948    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.767954    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768010    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768014    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768019    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768022    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768025    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768028    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:37:43.771993 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768031    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768033    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768036    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768039    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768041    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768044    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768047    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768049    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768052    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768054    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768057    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768059    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768062    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768066    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768070    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768072    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768076    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768079    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768083    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:37:43.772374 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768086    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768089    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768093    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768097    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768101    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768105    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768108    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768110    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768113    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768115    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768117    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768121    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768123    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768126    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768128    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768130    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768133    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768136    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768138    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:37:43.772896 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768141    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768143    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768146    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768148    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768151    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768153    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768156    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768158    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768161    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768163    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768166    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768169    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768171    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768173    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768176    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768178    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768181    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768184    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768186    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:37:43.773359 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768189    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768191    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768193    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768196    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768198    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768201    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768203    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768205    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768208    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768210    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768213    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768216    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768218    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768220    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768223    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768225    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768228    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768230    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768233    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:37:43.773827 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768235    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768238    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768240    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768242    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768245    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768247    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768250    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768252    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768255    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768257    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768259    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768262    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768264    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768267    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768269    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768271    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:37:43.774474 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768273    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.768278    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768385    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768389    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768392    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768395    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768397    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768399    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768402    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768405    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768407    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768409    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768412    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768414    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768417    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:37:43.775141 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768419    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768422    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768424    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768427    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768429    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768432    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768434    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768437    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768440    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768442    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768445    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768447    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768450    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768452    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768454    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768457    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768459    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768461    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768464    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768466    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:37:43.775537 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768469    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768471    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768474    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768477    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768479    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768482    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768484    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768486    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768489    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768491    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768494    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768497    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768499    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768501    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768504    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768506    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768509    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768512    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768514    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:37:43.776131 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768517    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768519    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768522    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768525    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768527    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768529    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768533    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768537    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768539    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768542    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768544    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768547    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768549    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768552    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768573    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768577    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768580    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768582    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768584    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:37:43.776670 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768587    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768591    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768594    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768597    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768600    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768604    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768607    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768609    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768612    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768614    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768617    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768619    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768622    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768624    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768627    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768629    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768632    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768635    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768637    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:37:43.777310 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768639    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768642    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768644    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768646    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768649    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768651    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768653    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768656    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:43.768658    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.768663    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.768765    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.772370    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.773954    2565 server.go:1019] "Starting client certificate rotation"
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.774075    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 20:37:43.777956 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.774119    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 20:37:43.778304 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.776351    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 20:37:43.778304 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.777588    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 20:37:43.784569 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.784537    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 10 20:37:43.786433 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.786419    2565 log.go:25] "Validated CRI v1 image API"
Jul 10 20:37:43.787079 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.787064    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 20:37:43.789275 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.789244    2565 fs.go:135] Filesystem UUIDs: map[0f7ea46b-e282-4fd4-8437-cf858a838867:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a6466829-5608-46b0-80d8-054adc11cb74:/dev/nvme0n1p3]
Jul 10 20:37:43.789370 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.789273    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 20:37:43.794302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.794187    2565 manager.go:216] Machine: {Timestamp:2026-07-10 20:37:43.793012593 +0000 UTC m=+0.285808762 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099788 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f977e94f03c2a2cf21189768ad270 SystemUUID:ec2f977e-94f0-3c2a-2cf2-1189768ad270 BootID:8a5ddfd6-a80a-4fa8-87f1-196c756a17cf Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cb:88:20:bc:7f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cb:88:20:bc:7f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:c3:9c:ad:01:1c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 20:37:43.794302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.794292    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 20:37:43.794475 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.794395    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 20:37:43.797006 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.796982    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 20:37:43.797166 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.797009    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-228.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 20:37:43.797708 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.797695    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 20:37:43.797766 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.797713    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 20:37:43.797766 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.797730    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 20:37:43.798316 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.798297    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 20:37:43.799214 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.799202    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 20:37:43.800074 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.800056    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 20:37:43.802996 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.802982    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 20:37:43.803133 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.803122    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 20:37:43.803198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.803160    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 20:37:43.803198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.803174    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 20:37:43.803198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.803197    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 20:37:43.803323 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.803210    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 10 20:37:43.803323 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.803221    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 20:37:43.803658 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.803645    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 20:37:43.803725 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.803668    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 20:37:43.805488 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.805473    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 20:37:43.808150 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.808126    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 20:37:43.808413 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.808395    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 20:37:43.809088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809075    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809094    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809103    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809113    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809122    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809128    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809135    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809140    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809146    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809152    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 20:37:43.809164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809169    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 20:37:43.809552 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.809174    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 20:37:43.810018 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.810006    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 20:37:43.810018 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.810016    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 20:37:43.812077 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.812050    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-228.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 20:37:43.812077 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.812068    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-228.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 20:37:43.812189 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.812050    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 20:37:43.813353 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.813340    2565 server.go:1268] "Started kubelet"
Jul 10 20:37:43.813433 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.813413    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 20:37:43.813481 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.813423    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 20:37:43.813530 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.813497    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 20:37:43.814367 ip-10-0-141-228 systemd[1]: Started Kubernetes Kubelet.
Jul 10 20:37:43.814709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.814691    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 20:37:43.814951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.814817    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 20:37:43.818857 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.818830    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 20:37:43.819122 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.819103    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 20:37:43.819122 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.819118    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 20:37:43.819271 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.819202    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:43.819750 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.819732    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 20:37:43.819750 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.819750    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 20:37:43.819917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.819839    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 20:37:43.819917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.819851    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 20:37:43.819917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.819882    2565 factory.go:55] Registering systemd factory
Jul 10 20:37:43.819917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.819898    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 10 20:37:43.820088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.819948    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 20:37:43.820135 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.820112    2565 factory.go:153] Registering CRI-O factory
Jul 10 20:37:43.820135 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.820124    2565 factory.go:223] Registration of the crio container factory successfully
Jul 10 20:37:43.820226 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.820176    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 20:37:43.820226 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.820200    2565 factory.go:103] Registering Raw factory
Jul 10 20:37:43.820226 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.820215    2565 manager.go:1196] Started watching for new ooms in manager
Jul 10 20:37:43.820543 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.820528    2565 manager.go:349] Starting recovery of all containers
Jul 10 20:37:43.829373 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.829219    2565 manager.go:354] Recovery completed
Jul 10 20:37:43.830042 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.830012    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 20:37:43.834081 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.834050    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-228.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 20:37:43.834882 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.834064    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-228.ec2.internal.18c10820aad24fad  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-228.ec2.internal,UID:ip-10-0-141-228.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-228.ec2.internal,},FirstTimestamp:2026-07-10 20:37:43.813312429 +0000 UTC m=+0.306108599,LastTimestamp:2026-07-10 20:37:43.813312429 +0000 UTC m=+0.306108599,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-228.ec2.internal,}"
Jul 10 20:37:43.835680 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.835667    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:37:43.837701 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.837682    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:37:43.837771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.837720    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:37:43.837771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.837734    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:37:43.838314 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.838297    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 20:37:43.838314 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.838312    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 20:37:43.838411 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.838328    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 20:37:43.838411 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.838312    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:37:43.839123 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.839054    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-228.ec2.internal.18c10820ac467681  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-228.ec2.internal,UID:ip-10-0-141-228.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-228.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-228.ec2.internal,},FirstTimestamp:2026-07-10 20:37:43.837701761 +0000 UTC m=+0.330497935,LastTimestamp:2026-07-10 20:37:43.837701761 +0000 UTC m=+0.330497935,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-228.ec2.internal,}"
Jul 10 20:37:43.839198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.839155    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:37:43.839198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.839183    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:37:43.839198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.839197    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:37:43.848303 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.840457    2565 policy_none.go:49] "None policy: Start"
Jul 10 20:37:43.848303 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.840473    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 20:37:43.848303 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.840483    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 20:37:43.848303 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.842464    2565 policy_none.go:47] "Start"
Jul 10 20:37:43.848498 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.848402    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rthgw"
Jul 10 20:37:43.849981 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.849907    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-228.ec2.internal.18c10820ac46dc3b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-228.ec2.internal,UID:ip-10-0-141-228.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-228.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-228.ec2.internal,},FirstTimestamp:2026-07-10 20:37:43.837727803 +0000 UTC m=+0.330523973,LastTimestamp:2026-07-10 20:37:43.837727803 +0000 UTC m=+0.330523973,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-228.ec2.internal,}"
Jul 10 20:37:43.853671 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.853615    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-228.ec2.internal.18c10820ac47042f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-228.ec2.internal,UID:ip-10-0-141-228.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-228.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-228.ec2.internal,},FirstTimestamp:2026-07-10 20:37:43.837738031 +0000 UTC m=+0.330534201,LastTimestamp:2026-07-10 20:37:43.837738031 +0000 UTC m=+0.330534201,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-228.ec2.internal,}"
Jul 10 20:37:43.855008 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.854989    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rthgw"
Jul 10 20:37:43.860159 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.860098    2565 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-228.ec2.internal.18c10820ac467681\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-228.ec2.internal.18c10820ac467681  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-228.ec2.internal,UID:ip-10-0-141-228.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-228.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-228.ec2.internal,},FirstTimestamp:2026-07-10 20:37:43.837701761 +0000 UTC m=+0.330497935,LastTimestamp:2026-07-10 20:37:43.839167837 +0000 UTC m=+0.331964007,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-228.ec2.internal,}"
Jul 10 20:37:43.886128 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.886107    2565 manager.go:337] "Starting Device Plugin manager"
Jul 10 20:37:43.886214 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.886161    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 20:37:43.886214 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.886173    2565 server.go:87] "Starting device plugin registration server"
Jul 10 20:37:43.886427 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.886416    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 20:37:43.886459 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.886430    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 20:37:43.886570 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.886533    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 20:37:43.886653 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.886639    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 20:37:43.886653 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.886648    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 20:37:43.887216 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.887191    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 20:37:43.887282 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.887237    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:43.949364 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.949332    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 20:37:43.951515 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.950642    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 20:37:43.951515 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.950677    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 20:37:43.951515 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.950699    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 20:37:43.951515 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.950734    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 20:37:43.953328 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.953310    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:37:43.987353 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.987329    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:37:43.988399 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.988382    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:37:43.988486 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.988418    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:37:43.988486 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.988433    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:37:43.988486 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.988473    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-228.ec2.internal"
Jul 10 20:37:43.997606 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:43.997588    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-228.ec2.internal"
Jul 10 20:37:43.997659 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:43.997609    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-228.ec2.internal\": node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:44.030226 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.030147    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:44.051505 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.051479    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"]
Jul 10 20:37:44.051636 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.051548    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:37:44.052463 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.052448    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:37:44.052536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.052482    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:37:44.052536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.052496    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:37:44.054733 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.054718    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:37:44.054848 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.054820    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.054893 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.054852    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:37:44.055421 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.055408    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:37:44.055467 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.055429    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:37:44.055467 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.055451    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:37:44.055522 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.055470    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:37:44.055522 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.055435    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:37:44.055607 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.055520    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:37:44.057710 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.057694    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.057762 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.057726    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:37:44.058390 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.058378    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:37:44.058446 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.058401    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:37:44.058446 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.058411    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:37:44.071079 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.071059    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-228.ec2.internal\" not found" node="ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.074788 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.074774    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-228.ec2.internal\" not found" node="ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.122301 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.122270    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7834fe1e11a9e3f6ca100a841c226e4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal\" (UID: \"b7834fe1e11a9e3f6ca100a841c226e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.122413 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.122304    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7834fe1e11a9e3f6ca100a841c226e4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal\" (UID: \"b7834fe1e11a9e3f6ca100a841c226e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.122413 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.122330    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3a5470e748fa4893099e28e4ea31d34-config\") pod \"kube-apiserver-proxy-ip-10-0-141-228.ec2.internal\" (UID: \"b3a5470e748fa4893099e28e4ea31d34\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.130259 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.130235    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:44.222704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.222676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7834fe1e11a9e3f6ca100a841c226e4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal\" (UID: \"b7834fe1e11a9e3f6ca100a841c226e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.222772 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.222710    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3a5470e748fa4893099e28e4ea31d34-config\") pod \"kube-apiserver-proxy-ip-10-0-141-228.ec2.internal\" (UID: \"b3a5470e748fa4893099e28e4ea31d34\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.222772 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.222732    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7834fe1e11a9e3f6ca100a841c226e4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal\" (UID: \"b7834fe1e11a9e3f6ca100a841c226e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.222838 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.222773    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7834fe1e11a9e3f6ca100a841c226e4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal\" (UID: \"b7834fe1e11a9e3f6ca100a841c226e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.222838 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.222792    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3a5470e748fa4893099e28e4ea31d34-config\") pod \"kube-apiserver-proxy-ip-10-0-141-228.ec2.internal\" (UID: \"b3a5470e748fa4893099e28e4ea31d34\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.222838 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.222806    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7834fe1e11a9e3f6ca100a841c226e4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal\" (UID: \"b7834fe1e11a9e3f6ca100a841c226e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.230785 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.230767    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:44.331682 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.331603    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:44.373816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.373788    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.377429 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.377410    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.432631 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.432598    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:44.533220 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.533188    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:44.633985 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.633924    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-228.ec2.internal\" not found"
Jul 10 20:37:44.634294 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.634279    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:37:44.634399 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.634385    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:37:44.719436 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.719408    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.727791 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.727767    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 20:37:44.729383 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.729370    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal"
Jul 10 20:37:44.741533 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.741508    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 20:37:44.775203 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.775178    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 20:37:44.775816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.775316    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 20:37:44.775816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.775352    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 20:37:44.803725 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.803688    2565 apiserver.go:52] "Watching apiserver"
Jul 10 20:37:44.813864 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.813835    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 20:37:44.816496 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.816458    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:37:44.816977 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.816916    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-ndzlv","kube-system/global-pull-secret-syncer-pdnrc","kube-system/konnectivity-agent-svbpz","kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx","openshift-cluster-node-tuning-operator/tuned-w5w2c","openshift-image-registry/node-ca-zhv8j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal","openshift-dns/node-resolver-9prbg","openshift-multus/multus-2bfwm","openshift-multus/multus-additional-cni-plugins-4slff","openshift-multus/network-metrics-daemon-nbqvl","openshift-network-diagnostics/network-check-target-w28x6","openshift-network-operator/iptables-alerter-pbth5"]
Jul 10 20:37:44.819236 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.819208    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 20:37:44.820238 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.820219    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.822681 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.822657    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:44.822779 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.822735    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:37:44.823758 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.823738    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-d4hb4\""
Jul 10 20:37:44.824103 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.824085    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 20:37:44.824169 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.824119    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 20:37:44.824377 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.824357    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 20:37:44.824426 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.824381    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 20:37:44.824981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.824962    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:37:44.826275 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826233    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-socket-dir-parent\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826375 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826309    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-run-k8s-cni-cncf-io\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826375 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826345    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-conf-dir\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826375 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826369    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-run-multus-certs\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826537 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826390    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-cni-dir\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826537 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826412    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-os-release\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826537 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826452    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-cni-binary-copy\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826537 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826469    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-run-netns\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826537 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826509    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-var-lib-kubelet\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826814 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-hostroot\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826814 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826591    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-system-cni-dir\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826814 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826619    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-var-lib-cni-bin\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826814 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826645    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-var-lib-cni-multus\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826814 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826672    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-daemon-config\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826814 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826695    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-etc-kubernetes\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826814 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826720    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-cnibin\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826814 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826743    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn48v\" (UniqueName: \"kubernetes.io/projected/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-kube-api-access-zn48v\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.826814 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.826766    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:44.827217 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.827198    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:44.827735 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.827719    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xbdqz\""
Jul 10 20:37:44.828076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.828057    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 20:37:44.828244 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.828231    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 20:37:44.829646 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.829631    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:44.830322 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.830303    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:37:44.830407 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.830329    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 20:37:44.830407 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.830378    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 20:37:44.830524 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.830378    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sgwbj\""
Jul 10 20:37:44.831867 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.831850    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.832003 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.831985    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87bjh\""
Jul 10 20:37:44.832396 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.832377    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 20:37:44.832453 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.832408    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 20:37:44.832494 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.832477    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 20:37:44.833091 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.833076    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 20:37:44.834187 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.834173    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-975lm\""
Jul 10 20:37:44.834268 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.834250    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:44.834370 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.834358    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:37:44.834487 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.834471    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 20:37:44.836694 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.836647    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:44.836955 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.836910    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 20:37:44.836955 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.836911    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4zph8\""
Jul 10 20:37:44.837136 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.837121    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 20:37:44.839069 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.839036    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:44.839158 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.839117    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:37:44.839224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.839141    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 20:37:44.839433 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.839407    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 20:37:44.839687 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.839670    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 20:37:44.839771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.839757    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tbns7\""
Jul 10 20:37:44.841485 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.841470    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.843960 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.843943    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:44.844110 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.844082    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 20:37:44.844170 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.844106    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 20:37:44.844285 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.844269    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 20:37:44.844348 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.844271    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 20:37:44.844348 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.844334    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f2hxl\""
Jul 10 20:37:44.844454 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.844408    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 20:37:44.845705 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.845691    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 20:37:44.846377 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.846363    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:44.846450 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.846434    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:37:44.846489 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.846441    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 20:37:44.847035 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.847018    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 20:37:44.847126 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.847107    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4f75q\""
Jul 10 20:37:44.859502 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.859457    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 20:32:44 +0000 UTC" deadline="2028-03-16 21:49:02.139770548 +0000 UTC"
Jul 10 20:37:44.859571 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.859507    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14761h11m17.280268373s"
Jul 10 20:37:44.865580 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.865553    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4qk5n"
Jul 10 20:37:44.879322 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.879306    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4qk5n"
Jul 10 20:37:44.921186 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.921166    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 20:37:44.927463 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927441    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-conf-dir\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.927548 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927473    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32d8f4e0-aad8-4d00-9f0b-52bc982481db-iptables-alerter-script\") pod \"iptables-alerter-pbth5\" (UID: \"32d8f4e0-aad8-4d00-9f0b-52bc982481db\") " pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:44.927548 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927492    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-run-systemd\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.927548 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927516    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:44.927700 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927578    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-conf-dir\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.927700 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927621    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-registration-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:44.927700 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927666    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-etc-selinux\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:44.927700 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927685    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32d8f4e0-aad8-4d00-9f0b-52bc982481db-host-slash\") pod \"iptables-alerter-pbth5\" (UID: \"32d8f4e0-aad8-4d00-9f0b-52bc982481db\") " pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:44.927862 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927703    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-var-lib-cni-multus\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.927862 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927721    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-daemon-config\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.927862 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927754    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-device-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:44.927862 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927774    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-lib-modules\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.927862 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927789    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-slash\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.927862 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927796    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-var-lib-cni-multus\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.927862 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927810    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-cni-netd\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.927862 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927835    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-ovn-node-metrics-cert\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927908    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d02870c-fd14-4cbd-b910-36cbb3f315b4-tmp\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927940    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsz8z\" (UniqueName: \"kubernetes.io/projected/a4629e74-b03f-44c1-b4b6-c46c6954b27a-kube-api-access-hsz8z\") pod \"node-resolver-9prbg\" (UID: \"a4629e74-b03f-44c1-b4b6-c46c6954b27a\") " pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927972    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn48v\" (UniqueName: \"kubernetes.io/projected/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-kube-api-access-zn48v\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.927991    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928028    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8n6d\" (UniqueName: \"kubernetes.io/projected/77bf1e8e-d631-4017-8dd2-7e8a4a094022-kube-api-access-f8n6d\") pod \"node-ca-zhv8j\" (UID: \"77bf1e8e-d631-4017-8dd2-7e8a4a094022\") " pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928078    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xn28\" (UniqueName: \"kubernetes.io/projected/5436825a-b217-4c2d-a358-97173d8da40a-kube-api-access-4xn28\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928105    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-systemd-units\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928131    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbhcf\" (UniqueName: \"kubernetes.io/projected/9d02870c-fd14-4cbd-b910-36cbb3f315b4-kube-api-access-wbhcf\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928157    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4629e74-b03f-44c1-b4b6-c46c6954b27a-tmp-dir\") pod \"node-resolver-9prbg\" (UID: \"a4629e74-b03f-44c1-b4b6-c46c6954b27a\") " pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:44.928197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-run-multus-certs\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928210    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-socket-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928250    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77bf1e8e-d631-4017-8dd2-7e8a4a094022-host\") pod \"node-ca-zhv8j\" (UID: \"77bf1e8e-d631-4017-8dd2-7e8a4a094022\") " pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928268    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-run-multus-certs\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928277    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-kubelet\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928302    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-system-cni-dir\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928330    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-cni-binary-copy\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928355    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-host\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928375    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-daemon-config\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928380    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-ovnkube-script-lib\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928404    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-etc-kubernetes\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-sysctl-d\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928482    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-etc-kubernetes\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.928497 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928497    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-run\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928531    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-sys\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928562    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-cnibin\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928617    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-modprobe-d\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928645    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4629e74-b03f-44c1-b4b6-c46c6954b27a-hosts-file\") pod \"node-resolver-9prbg\" (UID: \"a4629e74-b03f-44c1-b4b6-c46c6954b27a\") " pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928671    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80519b7a-2d8b-4103-9b9d-df134ba3845e-kubelet-config\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-cnibin\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-cni-binary-copy\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-socket-dir-parent\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928774    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkmjz\" (UniqueName: \"kubernetes.io/projected/b21894b3-163a-4b54-9247-6931e488ef66-kube-api-access-pkmjz\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928791    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/44d5f133-ec15-48c7-8fb5-26dbf7ce9106-agent-certs\") pod \"konnectivity-agent-svbpz\" (UID: \"44d5f133-ec15-48c7-8fb5-26dbf7ce9106\") " pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928790    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-socket-dir-parent\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928819    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-var-lib-openvswitch\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928842    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-ovnkube-config\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928863    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-cni-dir\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928879    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80519b7a-2d8b-4103-9b9d-df134ba3845e-dbus\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:44.929070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928912    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928927    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-multus-cni-dir\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928945    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-system-cni-dir\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.928974    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-var-lib-cni-bin\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929007    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-system-cni-dir\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-sys-fs\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929057    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-var-lib-cni-bin\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929098    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-sysctl-conf\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929124    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-run-openvswitch\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929142    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-cnibin\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929177    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-systemd\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929204    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-var-lib-kubelet\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929227    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929262    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929285    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/44d5f133-ec15-48c7-8fb5-26dbf7ce9106-konnectivity-ca\") pod \"konnectivity-agent-svbpz\" (UID: \"44d5f133-ec15-48c7-8fb5-26dbf7ce9106\") " pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929305    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cff8z\" (UniqueName: \"kubernetes.io/projected/32d8f4e0-aad8-4d00-9f0b-52bc982481db-kube-api-access-cff8z\") pod \"iptables-alerter-pbth5\" (UID: \"32d8f4e0-aad8-4d00-9f0b-52bc982481db\") " pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929346    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-kubernetes\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.929858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-log-socket\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929395    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-tuned\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-run-k8s-cni-cncf-io\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929457    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77bf1e8e-d631-4017-8dd2-7e8a4a094022-serviceca\") pod \"node-ca-zhv8j\" (UID: \"77bf1e8e-d631-4017-8dd2-7e8a4a094022\") " pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929465    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-run-k8s-cni-cncf-io\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929495    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-etc-openvswitch\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929521    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-os-release\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929544    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-cni-binary-copy\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929584    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-os-release\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929600    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-run-netns\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929622    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-var-lib-kubelet\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929629    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-os-release\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929646    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-hostroot\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929656    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-run-netns\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929673    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-host-var-lib-kubelet\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929671    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-sysconfig\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:44.930355 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929701    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-hostroot\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:44.930873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929712    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-run-netns\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.930873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929733    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-node-log\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.930873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929757    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-run-ovn-kubernetes\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.930873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929778    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-cni-bin\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.930873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929827    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9z9k\" (UniqueName: \"kubernetes.io/projected/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-kube-api-access-g9z9k\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:44.930873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929844    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-run-ovn\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.930873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929865    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.930873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-env-overrides\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.930873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.929904    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxr2z\" (UniqueName: \"kubernetes.io/projected/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-kube-api-access-cxr2z\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:44.934477 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.934454    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 20:37:44.934911 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.934891    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:37:44.935002 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.934918    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:37:44.935002 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.934931    2565 projected.go:196] Error preparing data for projected volume kube-api-access-wlwtm for pod openshift-network-diagnostics/network-check-target-w28x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:44.935114 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:44.935010    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm podName:48c57355-4382-4b5d-90de-faf573b05c48 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:45.434989694 +0000 UTC m=+1.927785875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wlwtm" (UniqueName: "kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm") pod "network-check-target-w28x6" (UID: "48c57355-4382-4b5d-90de-faf573b05c48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:44.937301 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:44.937283    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn48v\" (UniqueName: \"kubernetes.io/projected/e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900-kube-api-access-zn48v\") pod \"multus-2bfwm\" (UID: \"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900\") " pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:45.031157 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031110    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-registration-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.031157 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-etc-selinux\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031180    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32d8f4e0-aad8-4d00-9f0b-52bc982481db-host-slash\") pod \"iptables-alerter-pbth5\" (UID: \"32d8f4e0-aad8-4d00-9f0b-52bc982481db\") " pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031206    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-device-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031232    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-lib-modules\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031245    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-registration-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031283    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32d8f4e0-aad8-4d00-9f0b-52bc982481db-host-slash\") pod \"iptables-alerter-pbth5\" (UID: \"32d8f4e0-aad8-4d00-9f0b-52bc982481db\") " pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031304    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-device-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031322    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-etc-selinux\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031345    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-lib-modules\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031354    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-slash\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031330    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-slash\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031379    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-cni-netd\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031394    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-ovn-node-metrics-cert\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.031419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031412    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d02870c-fd14-4cbd-b910-36cbb3f315b4-tmp\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsz8z\" (UniqueName: \"kubernetes.io/projected/a4629e74-b03f-44c1-b4b6-c46c6954b27a-kube-api-access-hsz8z\") pod \"node-resolver-9prbg\" (UID: \"a4629e74-b03f-44c1-b4b6-c46c6954b27a\") " pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031440    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-cni-netd\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031477    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8n6d\" (UniqueName: \"kubernetes.io/projected/77bf1e8e-d631-4017-8dd2-7e8a4a094022-kube-api-access-f8n6d\") pod \"node-ca-zhv8j\" (UID: \"77bf1e8e-d631-4017-8dd2-7e8a4a094022\") " pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031501    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xn28\" (UniqueName: \"kubernetes.io/projected/5436825a-b217-4c2d-a358-97173d8da40a-kube-api-access-4xn28\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031525    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-systemd-units\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031550    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbhcf\" (UniqueName: \"kubernetes.io/projected/9d02870c-fd14-4cbd-b910-36cbb3f315b4-kube-api-access-wbhcf\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031572    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4629e74-b03f-44c1-b4b6-c46c6954b27a-tmp-dir\") pod \"node-resolver-9prbg\" (UID: \"a4629e74-b03f-44c1-b4b6-c46c6954b27a\") " pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031625    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-socket-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031658    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77bf1e8e-d631-4017-8dd2-7e8a4a094022-host\") pod \"node-ca-zhv8j\" (UID: \"77bf1e8e-d631-4017-8dd2-7e8a4a094022\") " pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031682    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-kubelet\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031700    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-systemd-units\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031705    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-system-cni-dir\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031747    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-system-cni-dir\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031766    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-host\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031804    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77bf1e8e-d631-4017-8dd2-7e8a4a094022-host\") pod \"node-ca-zhv8j\" (UID: \"77bf1e8e-d631-4017-8dd2-7e8a4a094022\") " pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031806    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-ovnkube-script-lib\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.032088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031857    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-socket-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031857    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-sysctl-d\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031898    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-run\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-sys\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031947    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-modprobe-d\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031963    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-sysctl-d\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031968    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-kubelet\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.031978    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4629e74-b03f-44c1-b4b6-c46c6954b27a-hosts-file\") pod \"node-resolver-9prbg\" (UID: \"a4629e74-b03f-44c1-b4b6-c46c6954b27a\") " pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032017    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80519b7a-2d8b-4103-9b9d-df134ba3845e-kubelet-config\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032028    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4629e74-b03f-44c1-b4b6-c46c6954b27a-hosts-file\") pod \"node-resolver-9prbg\" (UID: \"a4629e74-b03f-44c1-b4b6-c46c6954b27a\") " pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032049    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032066    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80519b7a-2d8b-4103-9b9d-df134ba3845e-kubelet-config\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032021    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-run\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032074    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4629e74-b03f-44c1-b4b6-c46c6954b27a-tmp-dir\") pod \"node-resolver-9prbg\" (UID: \"a4629e74-b03f-44c1-b4b6-c46c6954b27a\") " pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032118    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-modprobe-d\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032081    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkmjz\" (UniqueName: \"kubernetes.io/projected/b21894b3-163a-4b54-9247-6931e488ef66-kube-api-access-pkmjz\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032072    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-sys\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032125    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-host\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.032934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032192    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/44d5f133-ec15-48c7-8fb5-26dbf7ce9106-agent-certs\") pod \"konnectivity-agent-svbpz\" (UID: \"44d5f133-ec15-48c7-8fb5-26dbf7ce9106\") " pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032244    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-var-lib-openvswitch\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032364    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-ovnkube-config\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032393    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80519b7a-2d8b-4103-9b9d-df134ba3845e-dbus\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032409    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-var-lib-openvswitch\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032441    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032563    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-ovnkube-script-lib\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032686    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032811    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-sys-fs\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032869    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80519b7a-2d8b-4103-9b9d-df134ba3845e-dbus\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032890    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-sys-fs\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032912    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-sysctl-conf\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.032930    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032940    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-run-openvswitch\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032964    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-cnibin\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.032992    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-systemd\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033050    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-systemd\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033059    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-run-openvswitch\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.033849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033094    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-cnibin\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033073    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-sysctl-conf\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.033119    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret podName:80519b7a-2d8b-4103-9b9d-df134ba3845e nodeName:}" failed. No retries permitted until 2026-07-10 20:37:45.533008217 +0000 UTC m=+2.025804378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret") pod "global-pull-secret-syncer-pdnrc" (UID: "80519b7a-2d8b-4103-9b9d-df134ba3845e") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033143    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-var-lib-kubelet\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033201    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033228    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033252    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/44d5f133-ec15-48c7-8fb5-26dbf7ce9106-konnectivity-ca\") pod \"konnectivity-agent-svbpz\" (UID: \"44d5f133-ec15-48c7-8fb5-26dbf7ce9106\") " pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033256    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-var-lib-kubelet\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033290    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b21894b3-163a-4b54-9247-6931e488ef66-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033280    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cff8z\" (UniqueName: \"kubernetes.io/projected/32d8f4e0-aad8-4d00-9f0b-52bc982481db-kube-api-access-cff8z\") pod \"iptables-alerter-pbth5\" (UID: \"32d8f4e0-aad8-4d00-9f0b-52bc982481db\") " pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033338    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-kubernetes\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033364    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-log-socket\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.033381    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033389    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-tuned\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77bf1e8e-d631-4017-8dd2-7e8a4a094022-serviceca\") pod \"node-ca-zhv8j\" (UID: \"77bf1e8e-d631-4017-8dd2-7e8a4a094022\") " pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033442    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-ovnkube-config\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.034856 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.033468    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:37:45.533422959 +0000 UTC m=+2.026219137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033502    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-log-socket\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033543    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-kubernetes\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033684    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-etc-openvswitch\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033908    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77bf1e8e-d631-4017-8dd2-7e8a4a094022-serviceca\") pod \"node-ca-zhv8j\" (UID: \"77bf1e8e-d631-4017-8dd2-7e8a4a094022\") " pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033929    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/44d5f133-ec15-48c7-8fb5-26dbf7ce9106-konnectivity-ca\") pod \"konnectivity-agent-svbpz\" (UID: \"44d5f133-ec15-48c7-8fb5-26dbf7ce9106\") " pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.033510    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-etc-openvswitch\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034156    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-os-release\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034250    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-os-release\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034326    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-cni-binary-copy\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034362    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-sysconfig\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034442    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-run-netns\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-node-log\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034463    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d02870c-fd14-4cbd-b910-36cbb3f315b4-tmp\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034517    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-sysconfig\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034545    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-run-netns\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.035709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034550    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-run-ovn-kubernetes\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034573    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-run-ovn-kubernetes\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034601    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-cni-bin\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9z9k\" (UniqueName: \"kubernetes.io/projected/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-kube-api-access-g9z9k\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034670    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-run-ovn\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034701    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-env-overrides\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034755    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxr2z\" (UniqueName: \"kubernetes.io/projected/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-kube-api-access-cxr2z\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32d8f4e0-aad8-4d00-9f0b-52bc982481db-iptables-alerter-script\") pod \"iptables-alerter-pbth5\" (UID: \"32d8f4e0-aad8-4d00-9f0b-52bc982481db\") " pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034789    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-ovn-node-metrics-cert\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034808    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-run-systemd\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034841    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034900    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034930    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-cni-binary-copy\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.034954    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-host-cni-bin\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.035001    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-run-ovn\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.035021    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.036262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.035087    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.036805 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.035212    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-node-log\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036805 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.035270    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-run-systemd\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036805 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.035401    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-env-overrides\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.036805 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.035620    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32d8f4e0-aad8-4d00-9f0b-52bc982481db-iptables-alerter-script\") pod \"iptables-alerter-pbth5\" (UID: \"32d8f4e0-aad8-4d00-9f0b-52bc982481db\") " pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:45.036805 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.036300    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/44d5f133-ec15-48c7-8fb5-26dbf7ce9106-agent-certs\") pod \"konnectivity-agent-svbpz\" (UID: \"44d5f133-ec15-48c7-8fb5-26dbf7ce9106\") " pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:37:45.036805 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.036699    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9d02870c-fd14-4cbd-b910-36cbb3f315b4-etc-tuned\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.047966 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.047519    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsz8z\" (UniqueName: \"kubernetes.io/projected/a4629e74-b03f-44c1-b4b6-c46c6954b27a-kube-api-access-hsz8z\") pod \"node-resolver-9prbg\" (UID: \"a4629e74-b03f-44c1-b4b6-c46c6954b27a\") " pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:45.048400 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.048370    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkmjz\" (UniqueName: \"kubernetes.io/projected/b21894b3-163a-4b54-9247-6931e488ef66-kube-api-access-pkmjz\") pod \"aws-ebs-csi-driver-node-ktmqx\" (UID: \"b21894b3-163a-4b54-9247-6931e488ef66\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.048592 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.048571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbhcf\" (UniqueName: \"kubernetes.io/projected/9d02870c-fd14-4cbd-b910-36cbb3f315b4-kube-api-access-wbhcf\") pod \"tuned-w5w2c\" (UID: \"9d02870c-fd14-4cbd-b910-36cbb3f315b4\") " pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.048881 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.048858    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cff8z\" (UniqueName: \"kubernetes.io/projected/32d8f4e0-aad8-4d00-9f0b-52bc982481db-kube-api-access-cff8z\") pod \"iptables-alerter-pbth5\" (UID: \"32d8f4e0-aad8-4d00-9f0b-52bc982481db\") " pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:45.049003 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.048980    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxr2z\" (UniqueName: \"kubernetes.io/projected/00c5533a-15c8-4090-9d1a-e5e6ae42ff28-kube-api-access-cxr2z\") pod \"ovnkube-node-ndzlv\" (UID: \"00c5533a-15c8-4090-9d1a-e5e6ae42ff28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.049331 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.049305    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xn28\" (UniqueName: \"kubernetes.io/projected/5436825a-b217-4c2d-a358-97173d8da40a-kube-api-access-4xn28\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:45.050088 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.050068    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8n6d\" (UniqueName: \"kubernetes.io/projected/77bf1e8e-d631-4017-8dd2-7e8a4a094022-kube-api-access-f8n6d\") pod \"node-ca-zhv8j\" (UID: \"77bf1e8e-d631-4017-8dd2-7e8a4a094022\") " pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:45.050368 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.050351    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9z9k\" (UniqueName: \"kubernetes.io/projected/5918c8f3-28f5-411f-ab01-e7fa1c9e117c-kube-api-access-g9z9k\") pod \"multus-additional-cni-plugins-4slff\" (UID: \"5918c8f3-28f5-411f-ab01-e7fa1c9e117c\") " pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.103464 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.103418    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7834fe1e11a9e3f6ca100a841c226e4.slice/crio-e1644c6ff04c94db60bfc0ebe8abbc1f6b8b2be2163360c744cd40e2e13a8464 WatchSource:0}: Error finding container e1644c6ff04c94db60bfc0ebe8abbc1f6b8b2be2163360c744cd40e2e13a8464: Status 404 returned error can't find the container with id e1644c6ff04c94db60bfc0ebe8abbc1f6b8b2be2163360c744cd40e2e13a8464
Jul 10 20:37:45.103836 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.103804    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a5470e748fa4893099e28e4ea31d34.slice/crio-b996ac3a1d8cc9b20390bf5edff76a454f80f307710f8537fb9a09deefbe34e7 WatchSource:0}: Error finding container b996ac3a1d8cc9b20390bf5edff76a454f80f307710f8537fb9a09deefbe34e7: Status 404 returned error can't find the container with id b996ac3a1d8cc9b20390bf5edff76a454f80f307710f8537fb9a09deefbe34e7
Jul 10 20:37:45.108322 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.108307    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 20:37:45.158533 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.158453    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2bfwm"
Jul 10 20:37:45.162250 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.162224    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:37:45.169384 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.169360    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8085ca0_9a5f_4ca5_b7f6_adbf9fe71900.slice/crio-a72bbe83bf57a4272b8e0e3868adfe40370f077c08895825264ec80bdb024757 WatchSource:0}: Error finding container a72bbe83bf57a4272b8e0e3868adfe40370f077c08895825264ec80bdb024757: Status 404 returned error can't find the container with id a72bbe83bf57a4272b8e0e3868adfe40370f077c08895825264ec80bdb024757
Jul 10 20:37:45.171222 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.171202    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d5f133_ec15_48c7_8fb5_26dbf7ce9106.slice/crio-a1807328ebf462b4012d13e928c1427647ad5b67bef20811b236f0d91bfaa96f WatchSource:0}: Error finding container a1807328ebf462b4012d13e928c1427647ad5b67bef20811b236f0d91bfaa96f: Status 404 returned error can't find the container with id a1807328ebf462b4012d13e928c1427647ad5b67bef20811b236f0d91bfaa96f
Jul 10 20:37:45.182446 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.182427    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pbth5"
Jul 10 20:37:45.192957 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.192934    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d8f4e0_aad8_4d00_9f0b_52bc982481db.slice/crio-775754de43eaaa833394bcfb999700de34fa121c12f5d25597ddbfc395251168 WatchSource:0}: Error finding container 775754de43eaaa833394bcfb999700de34fa121c12f5d25597ddbfc395251168: Status 404 returned error can't find the container with id 775754de43eaaa833394bcfb999700de34fa121c12f5d25597ddbfc395251168
Jul 10 20:37:45.202403 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.202385    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx"
Jul 10 20:37:45.211821 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.211793    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb21894b3_163a_4b54_9247_6931e488ef66.slice/crio-3cee02a644f73ac6c4445f838b617ff71c2e08795e0e89edcb18f21fab168a72 WatchSource:0}: Error finding container 3cee02a644f73ac6c4445f838b617ff71c2e08795e0e89edcb18f21fab168a72: Status 404 returned error can't find the container with id 3cee02a644f73ac6c4445f838b617ff71c2e08795e0e89edcb18f21fab168a72
Jul 10 20:37:45.226826 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.226804    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5w2c"
Jul 10 20:37:45.234712 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.234691    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d02870c_fd14_4cbd_b910_36cbb3f315b4.slice/crio-ab33352af9e261d8fda661db7065bee916c1811c8b927d220c56f1a251165a76 WatchSource:0}: Error finding container ab33352af9e261d8fda661db7065bee916c1811c8b927d220c56f1a251165a76: Status 404 returned error can't find the container with id ab33352af9e261d8fda661db7065bee916c1811c8b927d220c56f1a251165a76
Jul 10 20:37:45.255735 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.255715    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9prbg"
Jul 10 20:37:45.263943 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.263916    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4629e74_b03f_44c1_b4b6_c46c6954b27a.slice/crio-851d0dc0c3f06e0eb7409903c9ef112d1ba90b4896922de48acea8a2362ddaa9 WatchSource:0}: Error finding container 851d0dc0c3f06e0eb7409903c9ef112d1ba90b4896922de48acea8a2362ddaa9: Status 404 returned error can't find the container with id 851d0dc0c3f06e0eb7409903c9ef112d1ba90b4896922de48acea8a2362ddaa9
Jul 10 20:37:45.274897 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.274879    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zhv8j"
Jul 10 20:37:45.283475 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.283454    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77bf1e8e_d631_4017_8dd2_7e8a4a094022.slice/crio-76ff8450e5c888d3b3203aeb920aab9a36cdc72dd4bfd5d9ca58f6b1e60b0caf WatchSource:0}: Error finding container 76ff8450e5c888d3b3203aeb920aab9a36cdc72dd4bfd5d9ca58f6b1e60b0caf: Status 404 returned error can't find the container with id 76ff8450e5c888d3b3203aeb920aab9a36cdc72dd4bfd5d9ca58f6b1e60b0caf
Jul 10 20:37:45.292538 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.292519    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:37:45.297351 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.297330    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4slff"
Jul 10 20:37:45.301791 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.301769    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c5533a_15c8_4090_9d1a_e5e6ae42ff28.slice/crio-be6ee2092bd8f6213f78199f131f7640514d4d3299fd995d8b2cedc8c978a5e5 WatchSource:0}: Error finding container be6ee2092bd8f6213f78199f131f7640514d4d3299fd995d8b2cedc8c978a5e5: Status 404 returned error can't find the container with id be6ee2092bd8f6213f78199f131f7640514d4d3299fd995d8b2cedc8c978a5e5
Jul 10 20:37:45.306541 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:37:45.306522    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5918c8f3_28f5_411f_ab01_e7fa1c9e117c.slice/crio-f9204fade15e55b1312cb58f07eba93e8658bd848ca9d742fecd61593a0495ce WatchSource:0}: Error finding container f9204fade15e55b1312cb58f07eba93e8658bd848ca9d742fecd61593a0495ce: Status 404 returned error can't find the container with id f9204fade15e55b1312cb58f07eba93e8658bd848ca9d742fecd61593a0495ce
Jul 10 20:37:45.308360 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.308335    2565 kuberuntime_manager.go:1459] "Unhandled Error" err="init container egress-router-binary-copy start failed in pod multus-additional-cni-plugins-4slff_openshift-multus(5918c8f3-28f5-411f-ab01-e7fa1c9e117c): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 10 20:37:45.308408 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.308381    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-4slff" podUID="5918c8f3-28f5-411f-ab01-e7fa1c9e117c"
Jul 10 20:37:45.436906 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.436812    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:45.437042 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.436975    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:37:45.437042 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.437008    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:37:45.437042 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.437019    2565 projected.go:196] Error preparing data for projected volume kube-api-access-wlwtm for pod openshift-network-diagnostics/network-check-target-w28x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:45.437143 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.437072    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm podName:48c57355-4382-4b5d-90de-faf573b05c48 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:46.437057087 +0000 UTC m=+2.929853246 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlwtm" (UniqueName: "kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm") pod "network-check-target-w28x6" (UID: "48c57355-4382-4b5d-90de-faf573b05c48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:45.537749 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.537702    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:45.537749 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.537764    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:45.538014 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.537899    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:45.538014 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.537900    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:45.538014 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.537971    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:37:46.537952398 +0000 UTC m=+3.030748562 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:45.538014 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.537991    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret podName:80519b7a-2d8b-4103-9b9d-df134ba3845e nodeName:}" failed. No retries permitted until 2026-07-10 20:37:46.537980653 +0000 UTC m=+3.030776815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret") pod "global-pull-secret-syncer-pdnrc" (UID: "80519b7a-2d8b-4103-9b9d-df134ba3845e") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:45.882973 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.882789    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:32:45 +0000 UTC" deadline="2028-03-10 18:29:16.194969546 +0000 UTC"
Jul 10 20:37:45.882973 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.882860    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14613h51m30.312114828s"
Jul 10 20:37:45.954810 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.954017    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:45.954810 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:45.954162    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:37:45.962815 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.962744    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx" event={"ID":"b21894b3-163a-4b54-9247-6931e488ef66","Type":"ContainerStarted","Data":"3cee02a644f73ac6c4445f838b617ff71c2e08795e0e89edcb18f21fab168a72"}
Jul 10 20:37:45.967365 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.967296    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pbth5" event={"ID":"32d8f4e0-aad8-4d00-9f0b-52bc982481db","Type":"ContainerStarted","Data":"775754de43eaaa833394bcfb999700de34fa121c12f5d25597ddbfc395251168"}
Jul 10 20:37:45.970578 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.970546    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-svbpz" event={"ID":"44d5f133-ec15-48c7-8fb5-26dbf7ce9106","Type":"ContainerStarted","Data":"a1807328ebf462b4012d13e928c1427647ad5b67bef20811b236f0d91bfaa96f"}
Jul 10 20:37:45.975827 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.975793    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zhv8j" event={"ID":"77bf1e8e-d631-4017-8dd2-7e8a4a094022","Type":"ContainerStarted","Data":"76ff8450e5c888d3b3203aeb920aab9a36cdc72dd4bfd5d9ca58f6b1e60b0caf"}
Jul 10 20:37:45.981763 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.981672    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9prbg" event={"ID":"a4629e74-b03f-44c1-b4b6-c46c6954b27a","Type":"ContainerStarted","Data":"851d0dc0c3f06e0eb7409903c9ef112d1ba90b4896922de48acea8a2362ddaa9"}
Jul 10 20:37:45.995020 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:45.994899    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5w2c" event={"ID":"9d02870c-fd14-4cbd-b910-36cbb3f315b4","Type":"ContainerStarted","Data":"ab33352af9e261d8fda661db7065bee916c1811c8b927d220c56f1a251165a76"}
Jul 10 20:37:46.024658 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.023452    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2bfwm" event={"ID":"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900","Type":"ContainerStarted","Data":"a72bbe83bf57a4272b8e0e3868adfe40370f077c08895825264ec80bdb024757"}
Jul 10 20:37:46.029397 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.029356    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal" event={"ID":"b3a5470e748fa4893099e28e4ea31d34","Type":"ContainerStarted","Data":"b996ac3a1d8cc9b20390bf5edff76a454f80f307710f8537fb9a09deefbe34e7"}
Jul 10 20:37:46.037871 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.037833    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal" event={"ID":"b7834fe1e11a9e3f6ca100a841c226e4","Type":"ContainerStarted","Data":"e1644c6ff04c94db60bfc0ebe8abbc1f6b8b2be2163360c744cd40e2e13a8464"}
Jul 10 20:37:46.043734 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.043703    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4slff" event={"ID":"5918c8f3-28f5-411f-ab01-e7fa1c9e117c","Type":"ContainerStarted","Data":"f9204fade15e55b1312cb58f07eba93e8658bd848ca9d742fecd61593a0495ce"}
Jul 10 20:37:46.049299 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.049264    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:852e5ae3f77e551118afa777a96222c9a7806f35ebea299f3e51e6fb499967c4\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-4slff" podUID="5918c8f3-28f5-411f-ab01-e7fa1c9e117c"
Jul 10 20:37:46.054420 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.054387    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerStarted","Data":"be6ee2092bd8f6213f78199f131f7640514d4d3299fd995d8b2cedc8c978a5e5"}
Jul 10 20:37:46.312073 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.311989    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:37:46.446574 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.445879    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:46.446574 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.446115    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:37:46.446574 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.446136    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:37:46.446574 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.446149    2565 projected.go:196] Error preparing data for projected volume kube-api-access-wlwtm for pod openshift-network-diagnostics/network-check-target-w28x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:46.446574 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.446211    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm podName:48c57355-4382-4b5d-90de-faf573b05c48 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:48.446192518 +0000 UTC m=+4.938988702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlwtm" (UniqueName: "kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm") pod "network-check-target-w28x6" (UID: "48c57355-4382-4b5d-90de-faf573b05c48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:46.492912 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.492620    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:37:46.547052 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.547014    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:46.547241 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.547073    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:46.547305 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.547266    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:46.547367 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.547330    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:37:48.547311005 +0000 UTC m=+5.040107167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:46.547800 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.547780    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:46.547894 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.547839    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret podName:80519b7a-2d8b-4103-9b9d-df134ba3845e nodeName:}" failed. No retries permitted until 2026-07-10 20:37:48.54782302 +0000 UTC m=+5.040619183 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret") pod "global-pull-secret-syncer-pdnrc" (UID: "80519b7a-2d8b-4103-9b9d-df134ba3845e") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:46.883262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.883197    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:32:45 +0000 UTC" deadline="2028-03-11 17:09:12.590860524 +0000 UTC"
Jul 10 20:37:46.883262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.883240    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14636h31m25.707626735s"
Jul 10 20:37:46.950972 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.950915    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:46.951152 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.951065    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:37:46.951561 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:46.951538    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:46.951689 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:46.951665    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:37:47.953948 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:47.953747    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:47.954490 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:47.954048    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:37:48.463133 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:48.463092    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:48.463315 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.463278    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:37:48.463315 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.463299    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:37:48.463315 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.463312    2565 projected.go:196] Error preparing data for projected volume kube-api-access-wlwtm for pod openshift-network-diagnostics/network-check-target-w28x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:48.463540 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.463377    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm podName:48c57355-4382-4b5d-90de-faf573b05c48 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:52.463355981 +0000 UTC m=+8.956152162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlwtm" (UniqueName: "kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm") pod "network-check-target-w28x6" (UID: "48c57355-4382-4b5d-90de-faf573b05c48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:48.563811 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:48.563771    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:48.563969 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:48.563828    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:48.564012 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.563989    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:48.564095 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.564057    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:37:52.56403657 +0000 UTC m=+9.056832732 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:48.564423 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.564404    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:48.564460 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.564452    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret podName:80519b7a-2d8b-4103-9b9d-df134ba3845e nodeName:}" failed. No retries permitted until 2026-07-10 20:37:52.564441721 +0000 UTC m=+9.057237884 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret") pod "global-pull-secret-syncer-pdnrc" (UID: "80519b7a-2d8b-4103-9b9d-df134ba3845e") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:48.952002 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:48.951922    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:48.952163 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.952094    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:37:48.952478 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:48.951928    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:48.952613 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:48.952562    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:37:49.953834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:49.953795    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:49.954275 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:49.953953    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:37:50.951376 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:50.951337    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:50.951555 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:50.951483    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:37:50.951732 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:50.951711    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:50.951829 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:50.951809    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:37:51.952487 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:51.951958    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:51.952487 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:51.952179    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:37:52.497043 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:52.496999    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:52.497239 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.497183    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:37:52.497239 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.497208    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:37:52.497239 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.497218    2565 projected.go:196] Error preparing data for projected volume kube-api-access-wlwtm for pod openshift-network-diagnostics/network-check-target-w28x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:52.497346 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.497274    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm podName:48c57355-4382-4b5d-90de-faf573b05c48 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:00.497254741 +0000 UTC m=+16.990050915 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlwtm" (UniqueName: "kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm") pod "network-check-target-w28x6" (UID: "48c57355-4382-4b5d-90de-faf573b05c48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:37:52.597541 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:52.597490    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:52.597731 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:52.597551    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:52.597731 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.597710    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:52.597848 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.597773    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:00.597753129 +0000 UTC m=+17.090549288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:37:52.598178 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.598158    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:52.598294 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.598214    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret podName:80519b7a-2d8b-4103-9b9d-df134ba3845e nodeName:}" failed. No retries permitted until 2026-07-10 20:38:00.598197772 +0000 UTC m=+17.090993933 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret") pod "global-pull-secret-syncer-pdnrc" (UID: "80519b7a-2d8b-4103-9b9d-df134ba3845e") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:37:52.951951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:52.951237    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:52.951951 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.951382    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:37:52.951951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:52.951767    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:52.951951 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:52.951859    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:37:53.952466 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:53.952008    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:53.952466 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:53.952118    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:37:54.951081 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:54.951047    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:54.951255 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:54.951198    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:37:54.951618 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:54.951592    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:54.951747 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:54.951682    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:37:55.951752 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:55.951719    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:55.952186 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:55.951864    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:37:56.952640 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:56.951495    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:56.952640 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:56.951787    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:56.952640 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:56.952322    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:37:56.952640 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:56.952530    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:37:57.951691 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:57.951654    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:57.951892 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:57.951780    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:37:58.951881 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:58.951844    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:37:58.952289 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:58.951860    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:37:58.952289 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:58.951970    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:37:58.952289 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:58.952039    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:37:59.951618 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:37:59.951085    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:37:59.951618 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:37:59.951227    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:00.560809 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:00.560780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:00.561155 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.560937    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:38:00.561155 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.560955    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:38:00.561155 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.560964    2565 projected.go:196] Error preparing data for projected volume kube-api-access-wlwtm for pod openshift-network-diagnostics/network-check-target-w28x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:38:00.561155 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.561014    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm podName:48c57355-4382-4b5d-90de-faf573b05c48 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:16.56099869 +0000 UTC m=+33.053794854 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlwtm" (UniqueName: "kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm") pod "network-check-target-w28x6" (UID: "48c57355-4382-4b5d-90de-faf573b05c48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:38:00.661278 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:00.661246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:00.661278 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:00.661281    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:00.661521 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.661390    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:38:00.661521 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.661435    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:38:00.661521 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.661453    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:16.661435454 +0000 UTC m=+33.154231612 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:38:00.661521 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.661510    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret podName:80519b7a-2d8b-4103-9b9d-df134ba3845e nodeName:}" failed. No retries permitted until 2026-07-10 20:38:16.661483422 +0000 UTC m=+33.154279592 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret") pod "global-pull-secret-syncer-pdnrc" (UID: "80519b7a-2d8b-4103-9b9d-df134ba3845e") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:38:00.951395 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:00.951312    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:00.951395 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:00.951347    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:00.951615 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.951444    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:00.951615 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:00.951598    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:01.951724 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:01.951686    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:01.952180 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:01.951850    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:02.951300 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:02.951268    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:02.951473 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:02.951268    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:02.951473 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:02.951403    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:02.951473 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:02.951464    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:03.952435 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:03.951949    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:03.952435 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:03.952071    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:04.951573 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:04.951528    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:04.951717 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:04.951658    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:04.951717 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:04.951706    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:04.951798 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:04.951785    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:05.952066 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:05.951836    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:05.952605 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:05.952123    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:06.111073 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.111032    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zhv8j" event={"ID":"77bf1e8e-d631-4017-8dd2-7e8a4a094022","Type":"ContainerStarted","Data":"233b04f51746493eaa931893a694053181f493ce977cb8a5a4fd8ef6cfe762a1"}
Jul 10 20:38:06.112635 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.112595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9prbg" event={"ID":"a4629e74-b03f-44c1-b4b6-c46c6954b27a","Type":"ContainerStarted","Data":"13347937c0daa094159086756b4421a3ae5ecd737b8ed56111fa9f3d8ed6610e"}
Jul 10 20:38:06.114095 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.114073    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5w2c" event={"ID":"9d02870c-fd14-4cbd-b910-36cbb3f315b4","Type":"ContainerStarted","Data":"f7404d554d455152ea52f37600e9d236aa94544c0ddebd9045242cca073349ab"}
Jul 10 20:38:06.119245 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.118662    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2bfwm" event={"ID":"e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900","Type":"ContainerStarted","Data":"92150ef9c26252b50ed595ee507eb69b1ed01b7201d1fcd636041883c9ffb17f"}
Jul 10 20:38:06.120922 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.120897    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal" event={"ID":"b3a5470e748fa4893099e28e4ea31d34","Type":"ContainerStarted","Data":"de1f56b7bf040318514e1bfc48317bfd2c5b7e0d76e361b434cab73d577cb620"}
Jul 10 20:38:06.122393 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.122373    2565 generic.go:356] "Generic (PLEG): container finished" podID="b7834fe1e11a9e3f6ca100a841c226e4" containerID="92e07bb73bf1b8d2a8cc75bac5388fb8e14b1fe2ba973b84b52c2b04e3cae0b1" exitCode=0
Jul 10 20:38:06.122483 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.122438    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal" event={"ID":"b7834fe1e11a9e3f6ca100a841c226e4","Type":"ContainerDied","Data":"92e07bb73bf1b8d2a8cc75bac5388fb8e14b1fe2ba973b84b52c2b04e3cae0b1"}
Jul 10 20:38:06.125767 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.125746    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:38:06.126133 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.126111    2565 generic.go:356] "Generic (PLEG): container finished" podID="00c5533a-15c8-4090-9d1a-e5e6ae42ff28" containerID="b6ec60789394379de857312ceb75580d435cbd8d883be6abf40a332b6ff517ef" exitCode=1
Jul 10 20:38:06.126194 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.126146    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerStarted","Data":"e2f825416574325bf17df0f772bde3ddaffb5dd8bcbb661f31ed83d29e2a9cc2"}
Jul 10 20:38:06.126241 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.126219    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerStarted","Data":"ec8e240535a11844414db55ac93cb5b2a0fd268f893bb60fc91563af470ce152"}
Jul 10 20:38:06.126284 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.126268    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerStarted","Data":"d0c7a677f21bf38952a957de1e7d5a4bef19ac8ad5f761f0bef26c77e71646fe"}
Jul 10 20:38:06.126324 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.126315    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerStarted","Data":"1e3f856cd865909db54dabdc1dcbbcab868d4521e13f6e35d8c86c9a65520c3b"}
Jul 10 20:38:06.126375 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.126362    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerDied","Data":"b6ec60789394379de857312ceb75580d435cbd8d883be6abf40a332b6ff517ef"}
Jul 10 20:38:06.126423 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.126415    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerStarted","Data":"6da0612f1f07d2ed4541506a5bccaa5a1d9275732710f100d41088dfd71cc6cb"}
Jul 10 20:38:06.127752 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.127730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx" event={"ID":"b21894b3-163a-4b54-9247-6931e488ef66","Type":"ContainerStarted","Data":"a00b46cac9471ed295d189d6ef4433a04a640683df70fce5a69799da89de5e2e"}
Jul 10 20:38:06.128250 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.128195    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zhv8j" podStartSLOduration=2.338786619 podStartE2EDuration="22.12817723s" totalImagesPullingTime="19.789390611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:06.127496674 +0000 UTC m=+22.620292854" watchObservedRunningTime="2026-07-10 20:38:06.12817723 +0000 UTC m=+22.620973448"
Jul 10 20:38:06.129031 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.129005    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-svbpz" event={"ID":"44d5f133-ec15-48c7-8fb5-26dbf7ce9106","Type":"ContainerStarted","Data":"2023f27c9b395aff5935068cc9c5792836cb15512ba6f97b0a09af3d24815a88"}
Jul 10 20:38:06.154927 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.154886    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-228.ec2.internal" podStartSLOduration=21.154874978 podStartE2EDuration="21.154874978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:06.140030529 +0000 UTC m=+22.632826708" watchObservedRunningTime="2026-07-10 20:38:06.154874978 +0000 UTC m=+22.647671157"
Jul 10 20:38:06.155043 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.155002    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w5w2c" podStartSLOduration=2.312233834 podStartE2EDuration="22.154994632s" totalImagesPullingTime="19.842760798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:06.154789342 +0000 UTC m=+22.647585525" watchObservedRunningTime="2026-07-10 20:38:06.154994632 +0000 UTC m=+22.647790813"
Jul 10 20:38:06.171014 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.170974    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9prbg" podStartSLOduration=2.35575764 podStartE2EDuration="22.170959978s" totalImagesPullingTime="19.815202338s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:06.170892901 +0000 UTC m=+22.663689081" watchObservedRunningTime="2026-07-10 20:38:06.170959978 +0000 UTC m=+22.663756138"
Jul 10 20:38:06.238265 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.238165    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2bfwm" podStartSLOduration=1.892515441 podStartE2EDuration="22.238149304s" totalImagesPullingTime="20.345633863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:06.222098339 +0000 UTC m=+22.714894519" watchObservedRunningTime="2026-07-10 20:38:06.238149304 +0000 UTC m=+22.730945475"
Jul 10 20:38:06.238671 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.238643    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-svbpz" podStartSLOduration=2.3366273189999998 podStartE2EDuration="22.238634018s" totalImagesPullingTime="19.902006699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:06.238127049 +0000 UTC m=+22.730923229" watchObservedRunningTime="2026-07-10 20:38:06.238634018 +0000 UTC m=+22.731430198"
Jul 10 20:38:06.951793 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.951763    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:06.951946 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:06.951889    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:06.951946 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:06.951899    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:06.952044 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:06.952029    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:07.132694 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.132597    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pbth5" event={"ID":"32d8f4e0-aad8-4d00-9f0b-52bc982481db","Type":"ContainerStarted","Data":"a72c7024e47d110ea0f6c972c46a00a099f5eb21553fd52a48992852056d7b34"}
Jul 10 20:38:07.134631 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.134605    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal" event={"ID":"b7834fe1e11a9e3f6ca100a841c226e4","Type":"ContainerStarted","Data":"cad8dade13dbb4c963b7fb7f4175ffeda2950dfa5cce054293653ae31045285d"}
Jul 10 20:38:07.136307 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.136279    2565 generic.go:356] "Generic (PLEG): container finished" podID="5918c8f3-28f5-411f-ab01-e7fa1c9e117c" containerID="143ecdc595d4f15b33f7140a5a4b4270b5004b95bb9d3f36d07a2a0d116bc49e" exitCode=0
Jul 10 20:38:07.136432 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.136366    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4slff" event={"ID":"5918c8f3-28f5-411f-ab01-e7fa1c9e117c","Type":"ContainerDied","Data":"143ecdc595d4f15b33f7140a5a4b4270b5004b95bb9d3f36d07a2a0d116bc49e"}
Jul 10 20:38:07.147327 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.147292    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pbth5" podStartSLOduration=3.296194941 podStartE2EDuration="23.147278055s" totalImagesPullingTime="19.851083114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:07.146679661 +0000 UTC m=+23.639475859" watchObservedRunningTime="2026-07-10 20:38:07.147278055 +0000 UTC m=+23.640074231"
Jul 10 20:38:07.181289 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.181239    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-228.ec2.internal" podStartSLOduration=22.181218984 podStartE2EDuration="22.181218984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:07.160719703 +0000 UTC m=+23.653515883" watchObservedRunningTime="2026-07-10 20:38:07.181218984 +0000 UTC m=+23.674015165"
Jul 10 20:38:07.642987 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.642954    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 20:38:07.900037 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.899927    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T20:38:07.642980331Z","UUID":"ff3102c0-a741-4563-834f-6f7e94c7f230","Handler":null,"Name":"","Endpoint":""}
Jul 10 20:38:07.901690 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.901662    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 20:38:07.901690 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.901697    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 20:38:07.951348 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:07.951318    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:07.951490 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:07.951427    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:08.145680 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:08.145648    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:38:08.147465 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:08.146273    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerStarted","Data":"797bfcfbf92d40a85d9b7b44c2e04ec68ed3dcd52c5b82dfb913c9fd9c26862d"}
Jul 10 20:38:08.149708 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:08.149106    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx" event={"ID":"b21894b3-163a-4b54-9247-6931e488ef66","Type":"ContainerStarted","Data":"6dbf660e4a9ddbc7957dee2d3dadf71d45eda4621d1f11bcde1537a6341cf23f"}
Jul 10 20:38:08.951633 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:08.951608    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:08.951787 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:08.951740    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:08.951892 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:08.951800    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:08.951955 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:08.951917    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:09.154409 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:09.154330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx" event={"ID":"b21894b3-163a-4b54-9247-6931e488ef66","Type":"ContainerStarted","Data":"f0d9c7018c81e4ab092622d128147385f2005c3ab326e18e9fd937523f733651"}
Jul 10 20:38:09.172285 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:09.172238    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ktmqx" podStartSLOduration=2.3292919579999998 podStartE2EDuration="25.172219906s" totalImagesPullingTime="22.842927948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:09.172163994 +0000 UTC m=+25.664960177" watchObservedRunningTime="2026-07-10 20:38:09.172219906 +0000 UTC m=+25.665016085"
Jul 10 20:38:09.951741 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:09.951715    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:09.951963 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:09.951824    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:10.877215 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:10.877172    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:38:10.877864 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:10.877844    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:38:10.951607 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:10.951550    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:10.951779 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:10.951552    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:10.951779 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:10.951705    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:10.951864 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:10.951788    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:11.158531 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:11.158445    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:38:11.159018 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:11.158998    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-svbpz"
Jul 10 20:38:11.952157 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:11.951916    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:11.952607 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:11.952172    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:12.162689 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:12.162653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4slff" event={"ID":"5918c8f3-28f5-411f-ab01-e7fa1c9e117c","Type":"ContainerStarted","Data":"658db44972836020fa1bffbef373321f0bc311b86420b7324713365b9c4a313b"}
Jul 10 20:38:12.167139 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:12.166401    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:38:12.167139 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:12.166875    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerStarted","Data":"bb04ee5d27620db67c267e6f413807929640af9f339eab508c9808f20d088ae9"}
Jul 10 20:38:12.167585 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:12.167546    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:38:12.167783 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:12.167766    2565 scope.go:117] "RemoveContainer" containerID="b6ec60789394379de857312ceb75580d435cbd8d883be6abf40a332b6ff517ef"
Jul 10 20:38:12.184530 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:12.184510    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:38:12.951585 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:12.951473    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:12.951769 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:12.951621    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:12.951769 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:12.951685    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:12.951856 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:12.951789    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:13.171467 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.171434    2565 generic.go:356] "Generic (PLEG): container finished" podID="5918c8f3-28f5-411f-ab01-e7fa1c9e117c" containerID="658db44972836020fa1bffbef373321f0bc311b86420b7324713365b9c4a313b" exitCode=0
Jul 10 20:38:13.172048 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.171504    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4slff" event={"ID":"5918c8f3-28f5-411f-ab01-e7fa1c9e117c","Type":"ContainerDied","Data":"658db44972836020fa1bffbef373321f0bc311b86420b7324713365b9c4a313b"}
Jul 10 20:38:13.179234 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.179209    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:38:13.179731 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.179705    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" event={"ID":"00c5533a-15c8-4090-9d1a-e5e6ae42ff28","Type":"ContainerStarted","Data":"c1063231464691c93bd9dabb1d29e4d68ff6248a43d5be179fce78ffbe50cd5f"}
Jul 10 20:38:13.179935 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.179914    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:38:13.180105 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.180092    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:38:13.196059 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.196030    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:38:13.221472 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.221392    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv" podStartSLOduration=9.393387967 podStartE2EDuration="29.221378791s" totalImagesPullingTime="19.827990824s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:13.219842127 +0000 UTC m=+29.712638318" watchObservedRunningTime="2026-07-10 20:38:13.221378791 +0000 UTC m=+29.714174969"
Jul 10 20:38:13.710143 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.710110    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pdnrc"]
Jul 10 20:38:13.710305 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.710233    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:13.710350 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:13.710310    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:13.713409 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.713382    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w28x6"]
Jul 10 20:38:13.713549 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.713470    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:13.713641 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:13.713597    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:13.714007 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.713986    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nbqvl"]
Jul 10 20:38:13.714096 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:13.714082    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:13.714206 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:13.714187    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:14.951870 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:14.951693    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:14.952445 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:14.951746    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:14.952445 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:14.951959    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:14.952445 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:14.952007    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:15.185457 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:15.185420    2565 generic.go:356] "Generic (PLEG): container finished" podID="5918c8f3-28f5-411f-ab01-e7fa1c9e117c" containerID="cd2b4f7cfbb7d9be7e386d8b5b8f99d4163f80d2f409daac31ee020473fff694" exitCode=0
Jul 10 20:38:15.185638 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:15.185490    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4slff" event={"ID":"5918c8f3-28f5-411f-ab01-e7fa1c9e117c","Type":"ContainerDied","Data":"cd2b4f7cfbb7d9be7e386d8b5b8f99d4163f80d2f409daac31ee020473fff694"}
Jul 10 20:38:15.951835 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:15.951804    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:15.951979 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:15.951906    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:16.584080 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:16.584054    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:16.584249 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.584230    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:38:16.584328 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.584259    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:38:16.584328 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.584271    2565 projected.go:196] Error preparing data for projected volume kube-api-access-wlwtm for pod openshift-network-diagnostics/network-check-target-w28x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:38:16.584415 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.584335    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm podName:48c57355-4382-4b5d-90de-faf573b05c48 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:48.584316408 +0000 UTC m=+65.077112579 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlwtm" (UniqueName: "kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm") pod "network-check-target-w28x6" (UID: "48c57355-4382-4b5d-90de-faf573b05c48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:38:16.684743 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:16.684714    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:16.684867 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:16.684759    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:16.684867 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.684854    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:38:16.684946 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.684875    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:38:16.684946 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.684915    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret podName:80519b7a-2d8b-4103-9b9d-df134ba3845e nodeName:}" failed. No retries permitted until 2026-07-10 20:38:48.684900724 +0000 UTC m=+65.177696882 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret") pod "global-pull-secret-syncer-pdnrc" (UID: "80519b7a-2d8b-4103-9b9d-df134ba3845e") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:38:16.684946 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.684931    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:48.684922577 +0000 UTC m=+65.177718733 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:38:16.951076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:16.951041    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:16.951076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:16.951076    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:16.951269 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.951171    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:38:16.951305 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:16.951280    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w28x6" podUID="48c57355-4382-4b5d-90de-faf573b05c48"
Jul 10 20:38:17.191816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:17.191784    2565 generic.go:356] "Generic (PLEG): container finished" podID="5918c8f3-28f5-411f-ab01-e7fa1c9e117c" containerID="778c0c45411f21e8866f2d8d940f10b268596bdfcfe59b4c6219df8b54e28ff6" exitCode=0
Jul 10 20:38:17.192223 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:17.191839    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4slff" event={"ID":"5918c8f3-28f5-411f-ab01-e7fa1c9e117c","Type":"ContainerDied","Data":"778c0c45411f21e8866f2d8d940f10b268596bdfcfe59b4c6219df8b54e28ff6"}
Jul 10 20:38:17.951300 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:17.951262    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:17.951455 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:17.951402    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdnrc" podUID="80519b7a-2d8b-4103-9b9d-df134ba3845e"
Jul 10 20:38:18.328532 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.328497    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-228.ec2.internal" event="NodeReady"
Jul 10 20:38:18.328988 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.328661    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 20:38:18.379282 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.379233    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"]
Jul 10 20:38:18.383585 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.383568    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bb489dc77-mfmzt"]
Jul 10 20:38:18.383719 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.383691    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:18.387496 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.387315    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 20:38:18.387598 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.387503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.392793 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.392773    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 20:38:18.393893 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.393877    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dtnv5\""
Jul 10 20:38:18.394525 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.394509    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 20:38:18.394653 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.394534    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ptttl\""
Jul 10 20:38:18.395108 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.395087    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 20:38:18.395206 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.395094    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 20:38:18.398570 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.398505    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"]
Jul 10 20:38:18.404473 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.404454    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t8kvw"]
Jul 10 20:38:18.408397 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.408378    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t8kvw"
Jul 10 20:38:18.409677 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.409657    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 20:38:18.411121 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.410891    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 20:38:18.411121 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.410998    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 20:38:18.411121 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.411038    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8wsvp\""
Jul 10 20:38:18.411121 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.410998    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 20:38:18.417998 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.417978    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t8kvw"]
Jul 10 20:38:18.420823 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.420805    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bb489dc77-mfmzt"]
Jul 10 20:38:18.478489 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.478459    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ts8f9"]
Jul 10 20:38:18.482218 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.482203    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.484778 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.484756    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 20:38:18.484864 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.484829    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 20:38:18.484917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.484899    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h798g\""
Jul 10 20:38:18.492063 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.490736    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ts8f9"]
Jul 10 20:38:18.501122 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501101    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-image-registry-private-configuration\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.501218 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501132    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:18.501218 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501170    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdtcn\" (UniqueName: \"kubernetes.io/projected/ff8e2022-8490-410b-968a-68c43e8d25f4-kube-api-access-hdtcn\") pod \"ingress-canary-t8kvw\" (UID: \"ff8e2022-8490-410b-968a-68c43e8d25f4\") " pod="openshift-ingress-canary/ingress-canary-t8kvw"
Jul 10 20:38:18.501218 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501197    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-installation-pull-secrets\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.501218 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501217    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:18.501369 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501234    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.501369 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501248    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e3380a-9607-4464-879c-f4bac8dff3c0-ca-trust-extracted\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.501369 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-certificates\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.501369 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501338    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2595s\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-kube-api-access-2595s\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.501369 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501361    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-bound-sa-token\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.501519 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501374    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff8e2022-8490-410b-968a-68c43e8d25f4-cert\") pod \"ingress-canary-t8kvw\" (UID: \"ff8e2022-8490-410b-968a-68c43e8d25f4\") " pod="openshift-ingress-canary/ingress-canary-t8kvw"
Jul 10 20:38:18.501519 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.501391    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-trusted-ca\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.602656 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.602569    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-certificates\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.602656 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.602622    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2595s\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-kube-api-access-2595s\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.602656 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.602649    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-bound-sa-token\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.602917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.602665    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff8e2022-8490-410b-968a-68c43e8d25f4-cert\") pod \"ingress-canary-t8kvw\" (UID: \"ff8e2022-8490-410b-968a-68c43e8d25f4\") " pod="openshift-ingress-canary/ingress-canary-t8kvw"
Jul 10 20:38:18.602917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.602692    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-trusted-ca\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.602917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.602717    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2265ffde-452f-4d13-9350-f0ba6bc280a2-config-volume\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.603059 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.602934    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-image-registry-private-configuration\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.603059 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.602978    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:18.603059 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603009    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzxpv\" (UniqueName: \"kubernetes.io/projected/2265ffde-452f-4d13-9350-f0ba6bc280a2-kube-api-access-vzxpv\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.603059 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603055    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdtcn\" (UniqueName: \"kubernetes.io/projected/ff8e2022-8490-410b-968a-68c43e8d25f4-kube-api-access-hdtcn\") pod \"ingress-canary-t8kvw\" (UID: \"ff8e2022-8490-410b-968a-68c43e8d25f4\") " pod="openshift-ingress-canary/ingress-canary-t8kvw"
Jul 10 20:38:18.603248 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603076    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.603248 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603100    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-installation-pull-secrets\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.603248 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603122    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:18.603248 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603142    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2265ffde-452f-4d13-9350-f0ba6bc280a2-tmp-dir\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.603248 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603170    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.603248 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603190    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e3380a-9607-4464-879c-f4bac8dff3c0-ca-trust-extracted\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.603248 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603239    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-certificates\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.603602 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:18.603455    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 20:38:18.603602 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603516    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e3380a-9607-4464-879c-f4bac8dff3c0-ca-trust-extracted\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.603602 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:18.603537    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert podName:eedc1267-4e6d-4dcf-b1eb-1232f2214f2d nodeName:}" failed. No retries permitted until 2026-07-10 20:38:19.103515678 +0000 UTC m=+35.596311838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wcxlk" (UID: "eedc1267-4e6d-4dcf-b1eb-1232f2214f2d") : secret "networking-console-plugin-cert" not found
Jul 10 20:38:18.603714 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:18.603626    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 20:38:18.603714 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:18.603640    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bb489dc77-mfmzt: secret "image-registry-tls" not found
Jul 10 20:38:18.603714 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603677    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-trusted-ca\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.603714 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:18.603694    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls podName:48e3380a-9607-4464-879c-f4bac8dff3c0 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:19.103679279 +0000 UTC m=+35.596475436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls") pod "image-registry-bb489dc77-mfmzt" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0") : secret "image-registry-tls" not found
Jul 10 20:38:18.603839 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.603759    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:18.607855 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.607682    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff8e2022-8490-410b-968a-68c43e8d25f4-cert\") pod \"ingress-canary-t8kvw\" (UID: \"ff8e2022-8490-410b-968a-68c43e8d25f4\") " pod="openshift-ingress-canary/ingress-canary-t8kvw"
Jul 10 20:38:18.607973 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.607689    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-image-registry-private-configuration\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.607973 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.607689    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-installation-pull-secrets\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.614347 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.614316    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-bound-sa-token\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.614515 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.614496    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2595s\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-kube-api-access-2595s\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:18.614817 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.614789    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdtcn\" (UniqueName: \"kubernetes.io/projected/ff8e2022-8490-410b-968a-68c43e8d25f4-kube-api-access-hdtcn\") pod \"ingress-canary-t8kvw\" (UID: \"ff8e2022-8490-410b-968a-68c43e8d25f4\") " pod="openshift-ingress-canary/ingress-canary-t8kvw"
Jul 10 20:38:18.703830 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.703787    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzxpv\" (UniqueName: \"kubernetes.io/projected/2265ffde-452f-4d13-9350-f0ba6bc280a2-kube-api-access-vzxpv\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.704002 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.703875    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.704002 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.703914    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2265ffde-452f-4d13-9350-f0ba6bc280a2-tmp-dir\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.704226 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:18.704206    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 20:38:18.704288 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.704266    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2265ffde-452f-4d13-9350-f0ba6bc280a2-config-volume\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.704341 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.704274    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2265ffde-452f-4d13-9350-f0ba6bc280a2-tmp-dir\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.704341 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:18.704295    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls podName:2265ffde-452f-4d13-9350-f0ba6bc280a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:19.204272969 +0000 UTC m=+35.697069140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls") pod "dns-default-ts8f9" (UID: "2265ffde-452f-4d13-9350-f0ba6bc280a2") : secret "dns-default-metrics-tls" not found
Jul 10 20:38:18.704723 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.704700    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2265ffde-452f-4d13-9350-f0ba6bc280a2-config-volume\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.713365 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.713338    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzxpv\" (UniqueName: \"kubernetes.io/projected/2265ffde-452f-4d13-9350-f0ba6bc280a2-kube-api-access-vzxpv\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:18.717967 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.717944    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t8kvw"
Jul 10 20:38:18.879605 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.879482    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t8kvw"]
Jul 10 20:38:18.883887 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:38:18.883864    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff8e2022_8490_410b_968a_68c43e8d25f4.slice/crio-9836447ff43fb7983fabc33405c3b60391d9200a742446d30e1de38b39599ed8 WatchSource:0}: Error finding container 9836447ff43fb7983fabc33405c3b60391d9200a742446d30e1de38b39599ed8: Status 404 returned error can't find the container with id 9836447ff43fb7983fabc33405c3b60391d9200a742446d30e1de38b39599ed8
Jul 10 20:38:18.951751 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.951694    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:18.951934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.951882    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:18.954650 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.954608    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 20:38:18.954769 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.954682    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 20:38:18.954769 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.954719    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n6zkh\""
Jul 10 20:38:18.954769 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.954761    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ggkq4\""
Jul 10 20:38:18.955006 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:18.954829    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 20:38:19.109714 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:19.109676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:19.109879 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:19.109729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:19.109879 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:19.109849    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 20:38:19.109879 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:19.109864    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bb489dc77-mfmzt: secret "image-registry-tls" not found
Jul 10 20:38:19.110047 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:19.109892    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 20:38:19.110047 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:19.109920    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls podName:48e3380a-9607-4464-879c-f4bac8dff3c0 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:20.109902503 +0000 UTC m=+36.602698661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls") pod "image-registry-bb489dc77-mfmzt" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0") : secret "image-registry-tls" not found
Jul 10 20:38:19.110047 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:19.109953    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert podName:eedc1267-4e6d-4dcf-b1eb-1232f2214f2d nodeName:}" failed. No retries permitted until 2026-07-10 20:38:20.109935715 +0000 UTC m=+36.602731874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wcxlk" (UID: "eedc1267-4e6d-4dcf-b1eb-1232f2214f2d") : secret "networking-console-plugin-cert" not found
Jul 10 20:38:19.198607 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:19.198501    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t8kvw" event={"ID":"ff8e2022-8490-410b-968a-68c43e8d25f4","Type":"ContainerStarted","Data":"9836447ff43fb7983fabc33405c3b60391d9200a742446d30e1de38b39599ed8"}
Jul 10 20:38:19.210915 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:19.210868    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:19.211060 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:19.211039    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 20:38:19.211131 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:19.211116    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls podName:2265ffde-452f-4d13-9350-f0ba6bc280a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:20.211098039 +0000 UTC m=+36.703894195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls") pod "dns-default-ts8f9" (UID: "2265ffde-452f-4d13-9350-f0ba6bc280a2") : secret "dns-default-metrics-tls" not found
Jul 10 20:38:19.951702 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:19.951670    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:19.954490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:19.954470    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 20:38:20.119172 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:20.119133    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:20.119347 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:20.119188    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:20.119401 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:20.119359    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 20:38:20.119401 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:20.119375    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bb489dc77-mfmzt: secret "image-registry-tls" not found
Jul 10 20:38:20.119483 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:20.119433    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls podName:48e3380a-9607-4464-879c-f4bac8dff3c0 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:22.119415718 +0000 UTC m=+38.612211877 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls") pod "image-registry-bb489dc77-mfmzt" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0") : secret "image-registry-tls" not found
Jul 10 20:38:20.119884 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:20.119861    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 20:38:20.120010 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:20.119928    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert podName:eedc1267-4e6d-4dcf-b1eb-1232f2214f2d nodeName:}" failed. No retries permitted until 2026-07-10 20:38:22.119907675 +0000 UTC m=+38.612703832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wcxlk" (UID: "eedc1267-4e6d-4dcf-b1eb-1232f2214f2d") : secret "networking-console-plugin-cert" not found
Jul 10 20:38:20.219973 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:20.219886    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:20.220122 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:20.220057    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 20:38:20.220194 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:20.220131    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls podName:2265ffde-452f-4d13-9350-f0ba6bc280a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:22.220109541 +0000 UTC m=+38.712905721 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls") pod "dns-default-ts8f9" (UID: "2265ffde-452f-4d13-9350-f0ba6bc280a2") : secret "dns-default-metrics-tls" not found
Jul 10 20:38:22.136733 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:22.136692    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:22.136733 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:22.136751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:22.137308 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:22.136872    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 20:38:22.137308 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:22.136948    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 20:38:22.137308 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:22.136968    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bb489dc77-mfmzt: secret "image-registry-tls" not found
Jul 10 20:38:22.137308 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:22.136958    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert podName:eedc1267-4e6d-4dcf-b1eb-1232f2214f2d nodeName:}" failed. No retries permitted until 2026-07-10 20:38:26.136936108 +0000 UTC m=+42.629732272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wcxlk" (UID: "eedc1267-4e6d-4dcf-b1eb-1232f2214f2d") : secret "networking-console-plugin-cert" not found
Jul 10 20:38:22.137308 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:22.137027    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls podName:48e3380a-9607-4464-879c-f4bac8dff3c0 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:26.137015205 +0000 UTC m=+42.629811362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls") pod "image-registry-bb489dc77-mfmzt" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0") : secret "image-registry-tls" not found
Jul 10 20:38:22.238350 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:22.237830    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:22.238350 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:22.238002    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 20:38:22.238350 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:22.238060    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls podName:2265ffde-452f-4d13-9350-f0ba6bc280a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:26.23804121 +0000 UTC m=+42.730837384 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls") pod "dns-default-ts8f9" (UID: "2265ffde-452f-4d13-9350-f0ba6bc280a2") : secret "dns-default-metrics-tls" not found
Jul 10 20:38:23.212529 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:23.212359    2565 generic.go:356] "Generic (PLEG): container finished" podID="5918c8f3-28f5-411f-ab01-e7fa1c9e117c" containerID="429e9dd0330308ba1eca7f419b4f891997ebd55025e959e5894a0a7af4636280" exitCode=0
Jul 10 20:38:23.213029 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:23.212428    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4slff" event={"ID":"5918c8f3-28f5-411f-ab01-e7fa1c9e117c","Type":"ContainerDied","Data":"429e9dd0330308ba1eca7f419b4f891997ebd55025e959e5894a0a7af4636280"}
Jul 10 20:38:23.214078 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:23.214056    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t8kvw" event={"ID":"ff8e2022-8490-410b-968a-68c43e8d25f4","Type":"ContainerStarted","Data":"21e822ac2df49a4812072828faf5dad78252c483c5eeab7190806627954caa02"}
Jul 10 20:38:23.248748 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:23.248703    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t8kvw" podStartSLOduration=1.6021277189999998 podStartE2EDuration="5.248683146s" totalImagesPullingTime="3.646555427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:23.248025218 +0000 UTC m=+39.740821399" watchObservedRunningTime="2026-07-10 20:38:23.248683146 +0000 UTC m=+39.741479328"
Jul 10 20:38:24.218517 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:24.218484    2565 generic.go:356] "Generic (PLEG): container finished" podID="5918c8f3-28f5-411f-ab01-e7fa1c9e117c" containerID="9d8ee4228e306b533bc8b1924591119af81ade0f48cac113f60418a2b691df7a" exitCode=0
Jul 10 20:38:24.219050 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:24.218553    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4slff" event={"ID":"5918c8f3-28f5-411f-ab01-e7fa1c9e117c","Type":"ContainerDied","Data":"9d8ee4228e306b533bc8b1924591119af81ade0f48cac113f60418a2b691df7a"}
Jul 10 20:38:25.224650 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:25.224615    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4slff" event={"ID":"5918c8f3-28f5-411f-ab01-e7fa1c9e117c","Type":"ContainerStarted","Data":"eca6cb3a6ef0963c94704fbe9447b6d543f4b42eb11bed2d8d8c1e350338d5e6"}
Jul 10 20:38:25.250652 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:25.250601    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4slff" podStartSLOduration=4.032474331 podStartE2EDuration="41.250587291s" totalImagesPullingTime="37.21811296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-10 20:38:25.249952694 +0000 UTC m=+41.742748873" watchObservedRunningTime="2026-07-10 20:38:25.250587291 +0000 UTC m=+41.743383469"
Jul 10 20:38:26.168117 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:26.168078    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:26.168117 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:26.168122    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:26.168369 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:26.168236    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 20:38:26.168369 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:26.168239    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 20:38:26.168369 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:26.168288    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bb489dc77-mfmzt: secret "image-registry-tls" not found
Jul 10 20:38:26.168369 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:26.168296    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert podName:eedc1267-4e6d-4dcf-b1eb-1232f2214f2d nodeName:}" failed. No retries permitted until 2026-07-10 20:38:34.168279577 +0000 UTC m=+50.661075734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wcxlk" (UID: "eedc1267-4e6d-4dcf-b1eb-1232f2214f2d") : secret "networking-console-plugin-cert" not found
Jul 10 20:38:26.168369 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:26.168337    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls podName:48e3380a-9607-4464-879c-f4bac8dff3c0 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:34.168321586 +0000 UTC m=+50.661117745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls") pod "image-registry-bb489dc77-mfmzt" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0") : secret "image-registry-tls" not found
Jul 10 20:38:26.268897 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:26.268862    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:26.269275 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:26.269002    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 20:38:26.269275 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:26.269059    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls podName:2265ffde-452f-4d13-9350-f0ba6bc280a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:34.26904433 +0000 UTC m=+50.761840487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls") pod "dns-default-ts8f9" (UID: "2265ffde-452f-4d13-9350-f0ba6bc280a2") : secret "dns-default-metrics-tls" not found
Jul 10 20:38:34.227732 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:34.227693    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:34.227732 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:34.227739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:34.228242 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:34.227861    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 20:38:34.228242 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:34.227861    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 20:38:34.228242 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:34.227871    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bb489dc77-mfmzt: secret "image-registry-tls" not found
Jul 10 20:38:34.228242 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:34.227930    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls podName:48e3380a-9607-4464-879c-f4bac8dff3c0 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:50.227914734 +0000 UTC m=+66.720710892 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls") pod "image-registry-bb489dc77-mfmzt" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0") : secret "image-registry-tls" not found
Jul 10 20:38:34.228242 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:34.227943    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert podName:eedc1267-4e6d-4dcf-b1eb-1232f2214f2d nodeName:}" failed. No retries permitted until 2026-07-10 20:38:50.227936582 +0000 UTC m=+66.720732739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wcxlk" (UID: "eedc1267-4e6d-4dcf-b1eb-1232f2214f2d") : secret "networking-console-plugin-cert" not found
Jul 10 20:38:34.328846 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:34.328809    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:34.329003 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:34.328951    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 20:38:34.329043 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:34.329012    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls podName:2265ffde-452f-4d13-9350-f0ba6bc280a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:50.328998238 +0000 UTC m=+66.821794398 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls") pod "dns-default-ts8f9" (UID: "2265ffde-452f-4d13-9350-f0ba6bc280a2") : secret "dns-default-metrics-tls" not found
Jul 10 20:38:45.196931 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:45.196896    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndzlv"
Jul 10 20:38:48.638230 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.638180    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:48.641209 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.641190    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 20:38:48.652109 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.652087    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 20:38:48.662665 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.662637    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlwtm\" (UniqueName: \"kubernetes.io/projected/48c57355-4382-4b5d-90de-faf573b05c48-kube-api-access-wlwtm\") pod \"network-check-target-w28x6\" (UID: \"48c57355-4382-4b5d-90de-faf573b05c48\") " pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:48.666179 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.666158    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ggkq4\""
Jul 10 20:38:48.674345 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.674328    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:48.738922 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.738896    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:48.739070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.738932    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:38:48.742064 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.742043    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 20:38:48.742193 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.742041    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 20:38:48.749976 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:48.749834    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 20:38:48.749976 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:48.749929    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:39:52.749903961 +0000 UTC m=+129.242700128 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : secret "metrics-daemon-secret" not found
Jul 10 20:38:48.752343 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.752319    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80519b7a-2d8b-4103-9b9d-df134ba3845e-original-pull-secret\") pod \"global-pull-secret-syncer-pdnrc\" (UID: \"80519b7a-2d8b-4103-9b9d-df134ba3845e\") " pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:48.763610 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.763547    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdnrc"
Jul 10 20:38:48.794717 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.794682    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w28x6"]
Jul 10 20:38:48.798625 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:38:48.798598    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c57355_4382_4b5d_90de_faf573b05c48.slice/crio-4845f26984e5db5e6309c63b8361cd4ea74ea3df589aa21bbc8b73c418c5cbb9 WatchSource:0}: Error finding container 4845f26984e5db5e6309c63b8361cd4ea74ea3df589aa21bbc8b73c418c5cbb9: Status 404 returned error can't find the container with id 4845f26984e5db5e6309c63b8361cd4ea74ea3df589aa21bbc8b73c418c5cbb9
Jul 10 20:38:48.886545 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:48.886513    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pdnrc"]
Jul 10 20:38:48.889457 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:38:48.889409    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80519b7a_2d8b_4103_9b9d_df134ba3845e.slice/crio-9bdcd5819aa044c8310c0c656213221f6a6a895e4761fa56166b84d57b05a3cd WatchSource:0}: Error finding container 9bdcd5819aa044c8310c0c656213221f6a6a895e4761fa56166b84d57b05a3cd: Status 404 returned error can't find the container with id 9bdcd5819aa044c8310c0c656213221f6a6a895e4761fa56166b84d57b05a3cd
Jul 10 20:38:49.281828 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:49.281743    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pdnrc" event={"ID":"80519b7a-2d8b-4103-9b9d-df134ba3845e","Type":"ContainerStarted","Data":"9bdcd5819aa044c8310c0c656213221f6a6a895e4761fa56166b84d57b05a3cd"}
Jul 10 20:38:49.282949 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:49.282919    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w28x6" event={"ID":"48c57355-4382-4b5d-90de-faf573b05c48","Type":"ContainerStarted","Data":"4845f26984e5db5e6309c63b8361cd4ea74ea3df589aa21bbc8b73c418c5cbb9"}
Jul 10 20:38:50.253887 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:50.253847    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:38:50.254334 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:50.253896    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:38:50.254334 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:50.254021    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 20:38:50.254334 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:50.254032    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bb489dc77-mfmzt: secret "image-registry-tls" not found
Jul 10 20:38:50.254334 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:50.254026    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 20:38:50.254334 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:50.254086    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls podName:48e3380a-9607-4464-879c-f4bac8dff3c0 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:22.25406971 +0000 UTC m=+98.746865867 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls") pod "image-registry-bb489dc77-mfmzt" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0") : secret "image-registry-tls" not found
Jul 10 20:38:50.254334 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:50.254103    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert podName:eedc1267-4e6d-4dcf-b1eb-1232f2214f2d nodeName:}" failed. No retries permitted until 2026-07-10 20:39:22.254094617 +0000 UTC m=+98.746890779 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wcxlk" (UID: "eedc1267-4e6d-4dcf-b1eb-1232f2214f2d") : secret "networking-console-plugin-cert" not found
Jul 10 20:38:50.354915 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:50.354877    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:38:50.355090 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:50.355049    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 20:38:50.355139 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:38:50.355130    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls podName:2265ffde-452f-4d13-9350-f0ba6bc280a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:22.355110064 +0000 UTC m=+98.847906224 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls") pod "dns-default-ts8f9" (UID: "2265ffde-452f-4d13-9350-f0ba6bc280a2") : secret "dns-default-metrics-tls" not found
Jul 10 20:38:52.292097 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:52.292014    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w28x6" event={"ID":"48c57355-4382-4b5d-90de-faf573b05c48","Type":"ContainerStarted","Data":"4d9fb544283892963e1ac696b0e0e9db0cbc921177d445471faf2b44a9f73037"}
Jul 10 20:38:52.292508 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:52.292150    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:38:52.309014 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:52.308962    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-w28x6" podStartSLOduration=65.24683046 podStartE2EDuration="1m8.308947498s" totalImagesPullingTime="3.062117038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:52.308004061 +0000 UTC m=+68.800800240" watchObservedRunningTime="2026-07-10 20:38:52.308947498 +0000 UTC m=+68.801743667"
Jul 10 20:38:54.300671 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:54.300629    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pdnrc" event={"ID":"80519b7a-2d8b-4103-9b9d-df134ba3845e","Type":"ContainerStarted","Data":"a8f7ec6cbe9ea009423a5674701f275287c0ee1054b75389f48d6281660c5597"}
Jul 10 20:38:54.316695 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:38:54.316633    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pdnrc" podStartSLOduration=65.853370987 podStartE2EDuration="1m10.316619172s" totalImagesPullingTime="4.463248185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:54.315287858 +0000 UTC m=+70.808084027" watchObservedRunningTime="2026-07-10 20:38:54.316619172 +0000 UTC m=+70.809415350"
Jul 10 20:39:22.290220 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:39:22.290076    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:39:22.290220 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:39:22.290121    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:39:22.290744 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:39:22.290235    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 20:39:22.290744 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:39:22.290243    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 20:39:22.290744 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:39:22.290262    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bb489dc77-mfmzt: secret "image-registry-tls" not found
Jul 10 20:39:22.290744 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:39:22.290311    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert podName:eedc1267-4e6d-4dcf-b1eb-1232f2214f2d nodeName:}" failed. No retries permitted until 2026-07-10 20:40:26.290296084 +0000 UTC m=+162.783092242 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wcxlk" (UID: "eedc1267-4e6d-4dcf-b1eb-1232f2214f2d") : secret "networking-console-plugin-cert" not found
Jul 10 20:39:22.290744 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:39:22.290327    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls podName:48e3380a-9607-4464-879c-f4bac8dff3c0 nodeName:}" failed. No retries permitted until 2026-07-10 20:40:26.29031998 +0000 UTC m=+162.783116136 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls") pod "image-registry-bb489dc77-mfmzt" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0") : secret "image-registry-tls" not found
Jul 10 20:39:22.390803 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:39:22.390763    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:39:22.391000 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:39:22.390935    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 20:39:22.391072 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:39:22.391025    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls podName:2265ffde-452f-4d13-9350-f0ba6bc280a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:40:26.391002328 +0000 UTC m=+162.883798486 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls") pod "dns-default-ts8f9" (UID: "2265ffde-452f-4d13-9350-f0ba6bc280a2") : secret "dns-default-metrics-tls" not found
Jul 10 20:39:23.299553 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:39:23.299521    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-w28x6"
Jul 10 20:39:52.810013 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:39:52.809961    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:39:52.810507 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:39:52.810122    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 20:39:52.810507 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:39:52.810237    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:41:54.810219008 +0000 UTC m=+251.303015168 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : secret "metrics-daemon-secret" not found
Jul 10 20:40:21.395886 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:21.395839    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk" podUID="eedc1267-4e6d-4dcf-b1eb-1232f2214f2d"
Jul 10 20:40:21.403443 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:21.403413    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt" podUID="48e3380a-9607-4464-879c-f4bac8dff3c0"
Jul 10 20:40:21.494369 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:21.494340    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ts8f9" podUID="2265ffde-452f-4d13-9350-f0ba6bc280a2"
Jul 10 20:40:21.500128 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:40:21.500105    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:40:21.500231 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:40:21.500113    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:40:21.500276 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:40:21.500115    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:40:21.969218 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:21.969180    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:40:26.354502 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:40:26.354460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:40:26.354502 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:40:26.354500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:40:26.354925 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:26.354642    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 20:40:26.354925 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:26.354719    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert podName:eedc1267-4e6d-4dcf-b1eb-1232f2214f2d nodeName:}" failed. No retries permitted until 2026-07-10 20:42:28.354694875 +0000 UTC m=+284.847491032 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wcxlk" (UID: "eedc1267-4e6d-4dcf-b1eb-1232f2214f2d") : secret "networking-console-plugin-cert" not found
Jul 10 20:40:26.354925 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:26.354646    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 20:40:26.354925 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:26.354738    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bb489dc77-mfmzt: secret "image-registry-tls" not found
Jul 10 20:40:26.354925 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:26.354777    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls podName:48e3380a-9607-4464-879c-f4bac8dff3c0 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:28.354765649 +0000 UTC m=+284.847561811 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls") pod "image-registry-bb489dc77-mfmzt" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0") : secret "image-registry-tls" not found
Jul 10 20:40:26.454848 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:40:26.454807    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:40:26.455024 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:26.454953    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 20:40:26.455024 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:40:26.455017    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls podName:2265ffde-452f-4d13-9350-f0ba6bc280a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:28.455002602 +0000 UTC m=+284.947798758 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls") pod "dns-default-ts8f9" (UID: "2265ffde-452f-4d13-9350-f0ba6bc280a2") : secret "dns-default-metrics-tls" not found
Jul 10 20:40:34.950938 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:40:34.950839    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:41:54.031891 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.031857    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"]
Jul 10 20:41:54.033602 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.033520    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.034830 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.034804    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"]
Jul 10 20:41:54.036676 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.036656    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-zxhjm"]
Jul 10 20:41:54.036827 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.036789    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:41:54.038376 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.038356    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.038842 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.038821    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4dxpm\""
Jul 10 20:41:54.039079 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.039063    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 10 20:41:54.039162 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.039080    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 10 20:41:54.039162 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.039122    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 10 20:41:54.039162 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.039063    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:41:54.039711 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.039694    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 10 20:41:54.043439 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.043420    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 10 20:41:54.043884 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.043870    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rmbzn\""
Jul 10 20:41:54.045815 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.045795    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:41:54.045914 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.045837    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-z4rng\""
Jul 10 20:41:54.048749 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.048729    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 10 20:41:54.048749 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.048742    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 10 20:41:54.048952 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.048934    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 10 20:41:54.049681 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.049665    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:41:54.055839 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.055819    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"]
Jul 10 20:41:54.057623 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.057608    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 10 20:41:54.067937 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.067918    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"]
Jul 10 20:41:54.079139 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.079120    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-zxhjm"]
Jul 10 20:41:54.129953 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.129918    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms"]
Jul 10 20:41:54.131765 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.131742    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms"
Jul 10 20:41:54.133737 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.133703    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-xhsh5"]
Jul 10 20:41:54.134390 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.134369    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:41:54.134483 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.134369    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7fh8k\""
Jul 10 20:41:54.134483 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.134369    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 10 20:41:54.135600 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.135584    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.138976 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.138955    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 20:41:54.139071 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.138979    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 10 20:41:54.139071 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.139025    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 10 20:41:54.139300 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.139282    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 20:41:54.139971 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.139955    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9ddj6\""
Jul 10 20:41:54.144073 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.144050    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms"]
Jul 10 20:41:54.146289 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.146270    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 10 20:41:54.152019 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.151998    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9d30526-a2fb-40a5-8894-3dbfe93639fd-serving-cert\") pod \"service-ca-operator-7695fc6c8c-22jqj\" (UID: \"e9d30526-a2fb-40a5-8894-3dbfe93639fd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.152101 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.152025    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc40c181-ad1c-4ef7-91cf-b35d8343f590-config\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.152101 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.152041    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc40c181-ad1c-4ef7-91cf-b35d8343f590-serving-cert\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.152164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.152115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzpzf\" (UniqueName: \"kubernetes.io/projected/cc40c181-ad1c-4ef7-91cf-b35d8343f590-kube-api-access-lzpzf\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.152200 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.152161    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc40c181-ad1c-4ef7-91cf-b35d8343f590-trusted-ca\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.152200 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.152185    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9d30526-a2fb-40a5-8894-3dbfe93639fd-config\") pod \"service-ca-operator-7695fc6c8c-22jqj\" (UID: \"e9d30526-a2fb-40a5-8894-3dbfe93639fd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.152282 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.152206    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77xlg\" (UniqueName: \"kubernetes.io/projected/e9d30526-a2fb-40a5-8894-3dbfe93639fd-kube-api-access-77xlg\") pod \"service-ca-operator-7695fc6c8c-22jqj\" (UID: \"e9d30526-a2fb-40a5-8894-3dbfe93639fd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.152333 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.152274    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:41:54.152333 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.152308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2gpr\" (UniqueName: \"kubernetes.io/projected/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-kube-api-access-t2gpr\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:41:54.152742 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.152726    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-xhsh5"]
Jul 10 20:41:54.252966 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.252932    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc40c181-ad1c-4ef7-91cf-b35d8343f590-config\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.253127 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.252974    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73cf4177-2d24-4f8f-a408-53dac7359398-tmp\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.253127 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.252991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/73cf4177-2d24-4f8f-a408-53dac7359398-snapshots\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.253127 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253011    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73cf4177-2d24-4f8f-a408-53dac7359398-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.253127 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253040    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9d30526-a2fb-40a5-8894-3dbfe93639fd-serving-cert\") pod \"service-ca-operator-7695fc6c8c-22jqj\" (UID: \"e9d30526-a2fb-40a5-8894-3dbfe93639fd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.253127 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253087    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw2cb\" (UniqueName: \"kubernetes.io/projected/b41db78f-c0d1-4449-9156-ce1651818206-kube-api-access-fw2cb\") pod \"volume-data-source-validator-78495597f6-rgbms\" (UID: \"b41db78f-c0d1-4449-9156-ce1651818206\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms"
Jul 10 20:41:54.253127 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253125    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc40c181-ad1c-4ef7-91cf-b35d8343f590-serving-cert\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.253341 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253164    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzpzf\" (UniqueName: \"kubernetes.io/projected/cc40c181-ad1c-4ef7-91cf-b35d8343f590-kube-api-access-lzpzf\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.253341 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253185    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73cf4177-2d24-4f8f-a408-53dac7359398-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.253341 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253213    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73cf4177-2d24-4f8f-a408-53dac7359398-serving-cert\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.253468 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253350    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc40c181-ad1c-4ef7-91cf-b35d8343f590-trusted-ca\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.253468 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253383    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9d30526-a2fb-40a5-8894-3dbfe93639fd-config\") pod \"service-ca-operator-7695fc6c8c-22jqj\" (UID: \"e9d30526-a2fb-40a5-8894-3dbfe93639fd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.253468 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253407    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77xlg\" (UniqueName: \"kubernetes.io/projected/e9d30526-a2fb-40a5-8894-3dbfe93639fd-kube-api-access-77xlg\") pod \"service-ca-operator-7695fc6c8c-22jqj\" (UID: \"e9d30526-a2fb-40a5-8894-3dbfe93639fd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.253468 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbrbn\" (UniqueName: \"kubernetes.io/projected/73cf4177-2d24-4f8f-a408-53dac7359398-kube-api-access-fbrbn\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.253468 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253462    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:41:54.253753 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253489    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2gpr\" (UniqueName: \"kubernetes.io/projected/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-kube-api-access-t2gpr\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:41:54.253753 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:54.253539    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 20:41:54.253753 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:54.253632    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls podName:99fdf4c5-d7f4-4be1-8f96-ed06d5982b19 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:54.753614208 +0000 UTC m=+251.246410366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls") pod "cluster-samples-operator-65db858894-c6t67" (UID: "99fdf4c5-d7f4-4be1-8f96-ed06d5982b19") : secret "samples-operator-tls" not found
Jul 10 20:41:54.253971 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.253947    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc40c181-ad1c-4ef7-91cf-b35d8343f590-config\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.254083 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.254057    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9d30526-a2fb-40a5-8894-3dbfe93639fd-config\") pod \"service-ca-operator-7695fc6c8c-22jqj\" (UID: \"e9d30526-a2fb-40a5-8894-3dbfe93639fd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.254758 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.254736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc40c181-ad1c-4ef7-91cf-b35d8343f590-trusted-ca\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.255551 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.255533    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc40c181-ad1c-4ef7-91cf-b35d8343f590-serving-cert\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.255648 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.255585    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9d30526-a2fb-40a5-8894-3dbfe93639fd-serving-cert\") pod \"service-ca-operator-7695fc6c8c-22jqj\" (UID: \"e9d30526-a2fb-40a5-8894-3dbfe93639fd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.260921 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.260895    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzpzf\" (UniqueName: \"kubernetes.io/projected/cc40c181-ad1c-4ef7-91cf-b35d8343f590-kube-api-access-lzpzf\") pod \"console-operator-7866497494-zxhjm\" (UID: \"cc40c181-ad1c-4ef7-91cf-b35d8343f590\") " pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.263485 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.263461    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77xlg\" (UniqueName: \"kubernetes.io/projected/e9d30526-a2fb-40a5-8894-3dbfe93639fd-kube-api-access-77xlg\") pod \"service-ca-operator-7695fc6c8c-22jqj\" (UID: \"e9d30526-a2fb-40a5-8894-3dbfe93639fd\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.267008 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.266980    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2gpr\" (UniqueName: \"kubernetes.io/projected/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-kube-api-access-t2gpr\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:41:54.343170 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.343148    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"
Jul 10 20:41:54.353800 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.353769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73cf4177-2d24-4f8f-a408-53dac7359398-tmp\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.353800 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.353800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/73cf4177-2d24-4f8f-a408-53dac7359398-snapshots\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.353971 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.353832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73cf4177-2d24-4f8f-a408-53dac7359398-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.353971 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.353839    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:41:54.354052 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.353941    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw2cb\" (UniqueName: \"kubernetes.io/projected/b41db78f-c0d1-4449-9156-ce1651818206-kube-api-access-fw2cb\") pod \"volume-data-source-validator-78495597f6-rgbms\" (UID: \"b41db78f-c0d1-4449-9156-ce1651818206\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms"
Jul 10 20:41:54.354104 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.354085    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73cf4177-2d24-4f8f-a408-53dac7359398-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.354153 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.354133    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73cf4177-2d24-4f8f-a408-53dac7359398-serving-cert\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.354203 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.354185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbrbn\" (UniqueName: \"kubernetes.io/projected/73cf4177-2d24-4f8f-a408-53dac7359398-kube-api-access-fbrbn\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.354406 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.354383    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73cf4177-2d24-4f8f-a408-53dac7359398-tmp\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.354527 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.354505    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/73cf4177-2d24-4f8f-a408-53dac7359398-snapshots\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.355023 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.354992    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73cf4177-2d24-4f8f-a408-53dac7359398-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.355491 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.355466    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73cf4177-2d24-4f8f-a408-53dac7359398-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.356772 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.356747    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73cf4177-2d24-4f8f-a408-53dac7359398-serving-cert\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.363237 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.363214    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw2cb\" (UniqueName: \"kubernetes.io/projected/b41db78f-c0d1-4449-9156-ce1651818206-kube-api-access-fw2cb\") pod \"volume-data-source-validator-78495597f6-rgbms\" (UID: \"b41db78f-c0d1-4449-9156-ce1651818206\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms"
Jul 10 20:41:54.363524 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.363507    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbrbn\" (UniqueName: \"kubernetes.io/projected/73cf4177-2d24-4f8f-a408-53dac7359398-kube-api-access-fbrbn\") pod \"insights-operator-68d9dd94cf-xhsh5\" (UID: \"73cf4177-2d24-4f8f-a408-53dac7359398\") " pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.442114 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.442078    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms"
Jul 10 20:41:54.449018 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.448994    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5"
Jul 10 20:41:54.475594 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.475547    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj"]
Jul 10 20:41:54.477936 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:41:54.477895    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d30526_a2fb_40a5_8894_3dbfe93639fd.slice/crio-8f415d4ac79d2ad468263ab0b19d979b0b9cc5b64d8ac50948a4d648e931cb63 WatchSource:0}: Error finding container 8f415d4ac79d2ad468263ab0b19d979b0b9cc5b64d8ac50948a4d648e931cb63: Status 404 returned error can't find the container with id 8f415d4ac79d2ad468263ab0b19d979b0b9cc5b64d8ac50948a4d648e931cb63
Jul 10 20:41:54.496721 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.496491    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-zxhjm"]
Jul 10 20:41:54.500435 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:41:54.500402    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc40c181_ad1c_4ef7_91cf_b35d8343f590.slice/crio-606beb74a1532fac6d8e0c84a5f2a1fe16c3ba4c4b3e1435bd7eb4a736ff738c WatchSource:0}: Error finding container 606beb74a1532fac6d8e0c84a5f2a1fe16c3ba4c4b3e1435bd7eb4a736ff738c: Status 404 returned error can't find the container with id 606beb74a1532fac6d8e0c84a5f2a1fe16c3ba4c4b3e1435bd7eb4a736ff738c
Jul 10 20:41:54.579875 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.579841    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms"]
Jul 10 20:41:54.582389 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:41:54.582359    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41db78f_c0d1_4449_9156_ce1651818206.slice/crio-8f11def8bd5f0c5051a084f62f53fabf1c2d9564602c470a16fbbb9b05d41f9d WatchSource:0}: Error finding container 8f11def8bd5f0c5051a084f62f53fabf1c2d9564602c470a16fbbb9b05d41f9d: Status 404 returned error can't find the container with id 8f11def8bd5f0c5051a084f62f53fabf1c2d9564602c470a16fbbb9b05d41f9d
Jul 10 20:41:54.595600 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.595535    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-xhsh5"]
Jul 10 20:41:54.598059 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:41:54.598037    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73cf4177_2d24_4f8f_a408_53dac7359398.slice/crio-78f42c7926f74d8f7646b84989264d516c43fd2e9cbf10e8855234c90dc8d631 WatchSource:0}: Error finding container 78f42c7926f74d8f7646b84989264d516c43fd2e9cbf10e8855234c90dc8d631: Status 404 returned error can't find the container with id 78f42c7926f74d8f7646b84989264d516c43fd2e9cbf10e8855234c90dc8d631
Jul 10 20:41:54.718112 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.718074    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj" event={"ID":"e9d30526-a2fb-40a5-8894-3dbfe93639fd","Type":"ContainerStarted","Data":"8f415d4ac79d2ad468263ab0b19d979b0b9cc5b64d8ac50948a4d648e931cb63"}
Jul 10 20:41:54.719131 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.719108    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5" event={"ID":"73cf4177-2d24-4f8f-a408-53dac7359398","Type":"ContainerStarted","Data":"78f42c7926f74d8f7646b84989264d516c43fd2e9cbf10e8855234c90dc8d631"}
Jul 10 20:41:54.720089 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.720070    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms" event={"ID":"b41db78f-c0d1-4449-9156-ce1651818206","Type":"ContainerStarted","Data":"8f11def8bd5f0c5051a084f62f53fabf1c2d9564602c470a16fbbb9b05d41f9d"}
Jul 10 20:41:54.721087 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.721067    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-zxhjm" event={"ID":"cc40c181-ad1c-4ef7-91cf-b35d8343f590","Type":"ContainerStarted","Data":"606beb74a1532fac6d8e0c84a5f2a1fe16c3ba4c4b3e1435bd7eb4a736ff738c"}
Jul 10 20:41:54.758614 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.758588    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:41:54.758752 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:54.758735    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 20:41:54.758802 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:54.758794    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls podName:99fdf4c5-d7f4-4be1-8f96-ed06d5982b19 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:55.758779976 +0000 UTC m=+252.251576132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls") pod "cluster-samples-operator-65db858894-c6t67" (UID: "99fdf4c5-d7f4-4be1-8f96-ed06d5982b19") : secret "samples-operator-tls" not found
Jul 10 20:41:54.859523 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:54.859451    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:41:54.859642 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:54.859576    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 20:41:54.859642 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:54.859623    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs podName:5436825a-b217-4c2d-a358-97173d8da40a nodeName:}" failed. No retries permitted until 2026-07-10 20:43:56.859610664 +0000 UTC m=+373.352406820 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs") pod "network-metrics-daemon-nbqvl" (UID: "5436825a-b217-4c2d-a358-97173d8da40a") : secret "metrics-daemon-secret" not found
Jul 10 20:41:55.768490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:55.768461    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:41:55.768972 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:55.768661    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 20:41:55.768972 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:55.768738    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls podName:99fdf4c5-d7f4-4be1-8f96-ed06d5982b19 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:57.768717768 +0000 UTC m=+254.261513935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls") pod "cluster-samples-operator-65db858894-c6t67" (UID: "99fdf4c5-d7f4-4be1-8f96-ed06d5982b19") : secret "samples-operator-tls" not found
Jul 10 20:41:57.788707 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:57.788676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:41:57.789039 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:57.788861    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 20:41:57.789039 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:57.788920    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls podName:99fdf4c5-d7f4-4be1-8f96-ed06d5982b19 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:01.78890449 +0000 UTC m=+258.281700648 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls") pod "cluster-samples-operator-65db858894-c6t67" (UID: "99fdf4c5-d7f4-4be1-8f96-ed06d5982b19") : secret "samples-operator-tls" not found
Jul 10 20:41:58.734488 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.734432    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj" event={"ID":"e9d30526-a2fb-40a5-8894-3dbfe93639fd","Type":"ContainerStarted","Data":"c2f57ec36c2e1975fb44614907a82ac6b0b73cf40a376c2393aa1f62a5cc24dc"}
Jul 10 20:41:58.735966 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.735940    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5" event={"ID":"73cf4177-2d24-4f8f-a408-53dac7359398","Type":"ContainerStarted","Data":"c66c17850cea952643f20dd56877f066b25ccc40a805c42e2e3a92134c314953"}
Jul 10 20:41:58.737392 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.737371    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms" event={"ID":"b41db78f-c0d1-4449-9156-ce1651818206","Type":"ContainerStarted","Data":"66a72f90586f1d27bd325ea6a12be03273d51cd6908c4405b6f6704bbe77abc5"}
Jul 10 20:41:58.738970 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.738952    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/0.log"
Jul 10 20:41:58.739040 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.738988    2565 generic.go:356] "Generic (PLEG): container finished" podID="cc40c181-ad1c-4ef7-91cf-b35d8343f590" containerID="df0993ce0b39e2d0dcb28ddde876e3811d589cb11d96560d9e3aefa4957adf73" exitCode=255
Jul 10 20:41:58.739040 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.739022    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-zxhjm" event={"ID":"cc40c181-ad1c-4ef7-91cf-b35d8343f590","Type":"ContainerDied","Data":"df0993ce0b39e2d0dcb28ddde876e3811d589cb11d96560d9e3aefa4957adf73"}
Jul 10 20:41:58.739195 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.739182    2565 scope.go:117] "RemoveContainer" containerID="df0993ce0b39e2d0dcb28ddde876e3811d589cb11d96560d9e3aefa4957adf73"
Jul 10 20:41:58.751308 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.751272    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj" podStartSLOduration=1.495386884 podStartE2EDuration="4.75125925s" totalImagesPullingTime="3.255872366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:41:58.750726239 +0000 UTC m=+255.243522422" watchObservedRunningTime="2026-07-10 20:41:58.75125925 +0000 UTC m=+255.244055429"
Jul 10 20:41:58.765657 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.765620    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rgbms" podStartSLOduration=1.612567055 podStartE2EDuration="4.765605453s" totalImagesPullingTime="3.153038398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:41:58.765451672 +0000 UTC m=+255.258247853" watchObservedRunningTime="2026-07-10 20:41:58.765605453 +0000 UTC m=+255.258401632"
Jul 10 20:41:58.782343 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.782285    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5" podStartSLOduration=1.643574657 podStartE2EDuration="4.782265099s" totalImagesPullingTime="3.138690442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:41:58.781637841 +0000 UTC m=+255.274434021" watchObservedRunningTime="2026-07-10 20:41:58.782265099 +0000 UTC m=+255.275061279"
Jul 10 20:41:58.974312 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.974278    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml"]
Jul 10 20:41:58.976282 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.976259    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml"
Jul 10 20:41:58.979238 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.978999    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 10 20:41:58.979238 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.979098    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gf7xp\""
Jul 10 20:41:58.979238 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.979077    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 10 20:41:58.986449 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:58.986385    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml"]
Jul 10 20:41:59.048034 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.047992    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pgsdg"]
Jul 10 20:41:59.049983 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.049960    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.053826 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.053805    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 20:41:59.053947 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.053893    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wt5hx\""
Jul 10 20:41:59.054601 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.054583    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 20:41:59.066594 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.066573    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pgsdg"]
Jul 10 20:41:59.099093 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.099060    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kshlz\" (UniqueName: \"kubernetes.io/projected/c8e66bf2-ca81-4e43-a8ee-6de156f4aee5-kube-api-access-kshlz\") pod \"migrator-7f59b54f6c-zn7ml\" (UID: \"c8e66bf2-ca81-4e43-a8ee-6de156f4aee5\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml"
Jul 10 20:41:59.200004 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.199972    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kshlz\" (UniqueName: \"kubernetes.io/projected/c8e66bf2-ca81-4e43-a8ee-6de156f4aee5-kube-api-access-kshlz\") pod \"migrator-7f59b54f6c-zn7ml\" (UID: \"c8e66bf2-ca81-4e43-a8ee-6de156f4aee5\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml"
Jul 10 20:41:59.200147 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.200008    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/50d32382-94d8-407b-aeef-4b0490d751bf-crio-socket\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.200147 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.200039    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/50d32382-94d8-407b-aeef-4b0490d751bf-data-volume\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.200227 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.200173    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.200264 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.200227    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.200309 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.200269    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn2r8\" (UniqueName: \"kubernetes.io/projected/50d32382-94d8-407b-aeef-4b0490d751bf-kube-api-access-dn2r8\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.207839 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.207815    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kshlz\" (UniqueName: \"kubernetes.io/projected/c8e66bf2-ca81-4e43-a8ee-6de156f4aee5-kube-api-access-kshlz\") pod \"migrator-7f59b54f6c-zn7ml\" (UID: \"c8e66bf2-ca81-4e43-a8ee-6de156f4aee5\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml"
Jul 10 20:41:59.287199 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.286976    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml"
Jul 10 20:41:59.301310 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.301279    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn2r8\" (UniqueName: \"kubernetes.io/projected/50d32382-94d8-407b-aeef-4b0490d751bf-kube-api-access-dn2r8\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.301439 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.301357    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/50d32382-94d8-407b-aeef-4b0490d751bf-crio-socket\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.301439 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.301390    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/50d32382-94d8-407b-aeef-4b0490d751bf-data-volume\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.301540 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.301447    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.301540 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.301475    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/50d32382-94d8-407b-aeef-4b0490d751bf-crio-socket\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.301540 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.301509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.301721 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:59.301615    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 20:41:59.301721 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:59.301634    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 20:41:59.301721 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:59.301694    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:41:59.801672772 +0000 UTC m=+256.294468954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : configmap "kube-rbac-proxy" not found
Jul 10 20:41:59.301721 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:59.301718    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:41:59.801706425 +0000 UTC m=+256.294502607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : secret "insights-runtime-extractor-tls" not found
Jul 10 20:41:59.301864 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.301781    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/50d32382-94d8-407b-aeef-4b0490d751bf-data-volume\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.310450 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.310425    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn2r8\" (UniqueName: \"kubernetes.io/projected/50d32382-94d8-407b-aeef-4b0490d751bf-kube-api-access-dn2r8\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.414129 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.414082    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml"]
Jul 10 20:41:59.417810 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:41:59.417784    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e66bf2_ca81_4e43_a8ee_6de156f4aee5.slice/crio-c9d88fbe56ba28881f3d98625f57143df03d40d195b1c4f6a436039321440836 WatchSource:0}: Error finding container c9d88fbe56ba28881f3d98625f57143df03d40d195b1c4f6a436039321440836: Status 404 returned error can't find the container with id c9d88fbe56ba28881f3d98625f57143df03d40d195b1c4f6a436039321440836
Jul 10 20:41:59.742960 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.742934    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/1.log"
Jul 10 20:41:59.743340 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.743325    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/0.log"
Jul 10 20:41:59.743392 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.743365    2565 generic.go:356] "Generic (PLEG): container finished" podID="cc40c181-ad1c-4ef7-91cf-b35d8343f590" containerID="84f243e2fd8d882950aa355cc577957c13908f6dc2d0d41e13c3562e76c248c5" exitCode=255
Jul 10 20:41:59.743441 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.743403    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-zxhjm" event={"ID":"cc40c181-ad1c-4ef7-91cf-b35d8343f590","Type":"ContainerDied","Data":"84f243e2fd8d882950aa355cc577957c13908f6dc2d0d41e13c3562e76c248c5"}
Jul 10 20:41:59.743491 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.743480    2565 scope.go:117] "RemoveContainer" containerID="df0993ce0b39e2d0dcb28ddde876e3811d589cb11d96560d9e3aefa4957adf73"
Jul 10 20:41:59.743683 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.743665    2565 scope.go:117] "RemoveContainer" containerID="84f243e2fd8d882950aa355cc577957c13908f6dc2d0d41e13c3562e76c248c5"
Jul 10 20:41:59.743853 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:59.743828    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-zxhjm_openshift-console-operator(cc40c181-ad1c-4ef7-91cf-b35d8343f590)\"" pod="openshift-console-operator/console-operator-7866497494-zxhjm" podUID="cc40c181-ad1c-4ef7-91cf-b35d8343f590"
Jul 10 20:41:59.744773 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.744574    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml" event={"ID":"c8e66bf2-ca81-4e43-a8ee-6de156f4aee5","Type":"ContainerStarted","Data":"c9d88fbe56ba28881f3d98625f57143df03d40d195b1c4f6a436039321440836"}
Jul 10 20:41:59.805647 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.805622    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.805768 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:41:59.805690    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:41:59.805768 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:59.805748    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 20:41:59.805845 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:59.805813    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:42:00.805795931 +0000 UTC m=+257.298592090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : configmap "kube-rbac-proxy" not found
Jul 10 20:41:59.805894 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:59.805841    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 20:41:59.805894 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:41:59.805887    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:42:00.805874193 +0000 UTC m=+257.298670349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : secret "insights-runtime-extractor-tls" not found
Jul 10 20:42:00.749980 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:00.749914    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/1.log"
Jul 10 20:42:00.750358 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:00.750251    2565 scope.go:117] "RemoveContainer" containerID="84f243e2fd8d882950aa355cc577957c13908f6dc2d0d41e13c3562e76c248c5"
Jul 10 20:42:00.750435 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:00.750409    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-zxhjm_openshift-console-operator(cc40c181-ad1c-4ef7-91cf-b35d8343f590)\"" pod="openshift-console-operator/console-operator-7866497494-zxhjm" podUID="cc40c181-ad1c-4ef7-91cf-b35d8343f590"
Jul 10 20:42:00.751789 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:00.751767    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml" event={"ID":"c8e66bf2-ca81-4e43-a8ee-6de156f4aee5","Type":"ContainerStarted","Data":"9092ab8b054e0369e45a397fcaae5030ac37c704a2e77094296f195770fb61da"}
Jul 10 20:42:00.751897 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:00.751817    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml" event={"ID":"c8e66bf2-ca81-4e43-a8ee-6de156f4aee5","Type":"ContainerStarted","Data":"a0536bf5407d2351d0c94c7b62673590a928c54fc9bb16178d3ad9714857c4f2"}
Jul 10 20:42:00.781546 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:00.781508    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zn7ml" podStartSLOduration=1.703602226 podStartE2EDuration="2.781495327s" totalImagesPullingTime="1.077893101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:00.781113912 +0000 UTC m=+257.273910088" watchObservedRunningTime="2026-07-10 20:42:00.781495327 +0000 UTC m=+257.274291505"
Jul 10 20:42:00.815505 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:00.815482    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:00.815629 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:00.815580    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:00.815680 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:00.815641    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 20:42:00.815680 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:00.815673    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 20:42:00.815753 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:00.815699    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:42:02.815684536 +0000 UTC m=+259.308480693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : secret "insights-runtime-extractor-tls" not found
Jul 10 20:42:00.815753 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:00.815716    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:42:02.815705159 +0000 UTC m=+259.308501315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : configmap "kube-rbac-proxy" not found
Jul 10 20:42:01.640702 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:01.640679    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9prbg_a4629e74-b03f-44c1-b4b6-c46c6954b27a/dns-node-resolver/0.log"
Jul 10 20:42:01.826764 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:01.826731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:42:01.827109 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:01.826834    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 20:42:01.827109 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:01.826886    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls podName:99fdf4c5-d7f4-4be1-8f96-ed06d5982b19 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:09.826872637 +0000 UTC m=+266.319668795 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls") pod "cluster-samples-operator-65db858894-c6t67" (UID: "99fdf4c5-d7f4-4be1-8f96-ed06d5982b19") : secret "samples-operator-tls" not found
Jul 10 20:42:02.835159 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:02.835119    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:02.835531 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:02.835182    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:02.835531 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:02.835248    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 20:42:02.835531 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:02.835271    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 20:42:02.835531 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:02.835325    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:42:06.835305233 +0000 UTC m=+263.328101392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : configmap "kube-rbac-proxy" not found
Jul 10 20:42:02.835531 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:02.835344    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:42:06.83533216 +0000 UTC m=+263.328128329 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : secret "insights-runtime-extractor-tls" not found
Jul 10 20:42:03.039458 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:03.039428    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zhv8j_77bf1e8e-d631-4017-8dd2-7e8a4a094022/node-ca/0.log"
Jul 10 20:42:03.641768 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:03.641740    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t8kvw_ff8e2022-8490-410b-968a-68c43e8d25f4/serve-healthcheck-canary/0.log"
Jul 10 20:42:04.040520 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:04.040407    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-zn7ml_c8e66bf2-ca81-4e43-a8ee-6de156f4aee5/migrator/0.log"
Jul 10 20:42:04.239817 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:04.239763    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-zn7ml_c8e66bf2-ca81-4e43-a8ee-6de156f4aee5/graceful-termination/0.log"
Jul 10 20:42:04.354808 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:04.354778    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:42:04.354959 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:04.354830    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:42:04.355141 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:04.355129    2565 scope.go:117] "RemoveContainer" containerID="84f243e2fd8d882950aa355cc577957c13908f6dc2d0d41e13c3562e76c248c5"
Jul 10 20:42:04.355286 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:04.355269    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-zxhjm_openshift-console-operator(cc40c181-ad1c-4ef7-91cf-b35d8343f590)\"" pod="openshift-console-operator/console-operator-7866497494-zxhjm" podUID="cc40c181-ad1c-4ef7-91cf-b35d8343f590"
Jul 10 20:42:06.868429 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:06.868375    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:06.868908 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:06.868483    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:06.868908 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:06.868551    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 20:42:06.868908 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:06.868576    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 20:42:06.868908 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:06.868622    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:42:14.868607125 +0000 UTC m=+271.361403282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : configmap "kube-rbac-proxy" not found
Jul 10 20:42:06.868908 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:06.868662    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:42:14.868642794 +0000 UTC m=+271.361438974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : secret "insights-runtime-extractor-tls" not found
Jul 10 20:42:09.894677 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:09.894628    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:42:09.897129 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:09.897098    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/99fdf4c5-d7f4-4be1-8f96-ed06d5982b19-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-c6t67\" (UID: \"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:42:09.948996 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:09.948964    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"
Jul 10 20:42:10.077926 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:10.077890    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67"]
Jul 10 20:42:10.784421 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:10.784386    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67" event={"ID":"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19","Type":"ContainerStarted","Data":"c3e328ae8452096c4982133104b055103b636b71243feca632b407985e0eb1a1"}
Jul 10 20:42:12.793456 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:12.793415    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67" event={"ID":"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19","Type":"ContainerStarted","Data":"47cd45787bc814564bc3f94b9cb7c15bbd6bef5e9370748350a3a6f24363053a"}
Jul 10 20:42:12.793813 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:12.793479    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67" event={"ID":"99fdf4c5-d7f4-4be1-8f96-ed06d5982b19","Type":"ContainerStarted","Data":"019fae90f2ed9206860c7085b5a3130e62cac87f3d352604851a3c5c9d3676f2"}
Jul 10 20:42:12.811362 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:12.811316    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-c6t67" podStartSLOduration=17.182964733 podStartE2EDuration="18.811301862s" totalImagesPullingTime="1.628337129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:12.810779093 +0000 UTC m=+269.303575271" watchObservedRunningTime="2026-07-10 20:42:12.811301862 +0000 UTC m=+269.304098076"
Jul 10 20:42:14.936623 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:14.936586    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:14.937113 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:14.936673    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:14.937113 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:14.936724    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 20:42:14.937113 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:14.936794    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls podName:50d32382-94d8-407b-aeef-4b0490d751bf nodeName:}" failed. No retries permitted until 2026-07-10 20:42:30.93677898 +0000 UTC m=+287.429575137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgsdg" (UID: "50d32382-94d8-407b-aeef-4b0490d751bf") : secret "insights-runtime-extractor-tls" not found
Jul 10 20:42:14.937113 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:14.937103    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/50d32382-94d8-407b-aeef-4b0490d751bf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:17.952069 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:17.952030    2565 scope.go:117] "RemoveContainer" containerID="84f243e2fd8d882950aa355cc577957c13908f6dc2d0d41e13c3562e76c248c5"
Jul 10 20:42:18.812337 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:18.812310    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:42:18.812750 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:18.812734    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/1.log"
Jul 10 20:42:18.812830 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:18.812778    2565 generic.go:356] "Generic (PLEG): container finished" podID="cc40c181-ad1c-4ef7-91cf-b35d8343f590" containerID="b36a29b165e1d9a0336701e828e3ba8223927b7354c0c9e53c86abce88b522d0" exitCode=255
Jul 10 20:42:18.812883 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:18.812830    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-zxhjm" event={"ID":"cc40c181-ad1c-4ef7-91cf-b35d8343f590","Type":"ContainerDied","Data":"b36a29b165e1d9a0336701e828e3ba8223927b7354c0c9e53c86abce88b522d0"}
Jul 10 20:42:18.812883 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:18.812878    2565 scope.go:117] "RemoveContainer" containerID="84f243e2fd8d882950aa355cc577957c13908f6dc2d0d41e13c3562e76c248c5"
Jul 10 20:42:18.813190 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:18.813173    2565 scope.go:117] "RemoveContainer" containerID="b36a29b165e1d9a0336701e828e3ba8223927b7354c0c9e53c86abce88b522d0"
Jul 10 20:42:18.813387 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:18.813362    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-zxhjm_openshift-console-operator(cc40c181-ad1c-4ef7-91cf-b35d8343f590)\"" pod="openshift-console-operator/console-operator-7866497494-zxhjm" podUID="cc40c181-ad1c-4ef7-91cf-b35d8343f590"
Jul 10 20:42:19.816684 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:19.816655    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:42:24.354732 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:24.354697    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:42:24.355129 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:24.354754    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:42:24.355129 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:24.355037    2565 scope.go:117] "RemoveContainer" containerID="b36a29b165e1d9a0336701e828e3ba8223927b7354c0c9e53c86abce88b522d0"
Jul 10 20:42:24.355204 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:24.355185    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-zxhjm_openshift-console-operator(cc40c181-ad1c-4ef7-91cf-b35d8343f590)\"" pod="openshift-console-operator/console-operator-7866497494-zxhjm" podUID="cc40c181-ad1c-4ef7-91cf-b35d8343f590"
Jul 10 20:42:24.500671 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:24.500631    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ts8f9" podUID="2265ffde-452f-4d13-9350-f0ba6bc280a2"
Jul 10 20:42:24.500671 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:24.500646    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt" podUID="48e3380a-9607-4464-879c-f4bac8dff3c0"
Jul 10 20:42:24.500873 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:24.500646    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk" podUID="eedc1267-4e6d-4dcf-b1eb-1232f2214f2d"
Jul 10 20:42:24.833116 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:24.833088    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:42:24.833267 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:24.833090    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:42:24.833267 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:24.833095    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:42:28.442456 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.442419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:42:28.442835 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.442466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:42:28.444974 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.444950    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eedc1267-4e6d-4dcf-b1eb-1232f2214f2d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wcxlk\" (UID: \"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:42:28.444974 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.444964    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"image-registry-bb489dc77-mfmzt\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") " pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:42:28.543300 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.543255    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:42:28.545702 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.545683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2265ffde-452f-4d13-9350-f0ba6bc280a2-metrics-tls\") pod \"dns-default-ts8f9\" (UID: \"2265ffde-452f-4d13-9350-f0ba6bc280a2\") " pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:42:28.737371 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.737280    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h798g\""
Jul 10 20:42:28.737371 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.737282    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dtnv5\""
Jul 10 20:42:28.737371 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.737282    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ptttl\""
Jul 10 20:42:28.743616 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.743600    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:42:28.743697 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.743680    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"
Jul 10 20:42:28.743740 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.743718    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:42:28.889090 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.889064    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wcxlk"]
Jul 10 20:42:28.891663 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:28.891629    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeedc1267_4e6d_4dcf_b1eb_1232f2214f2d.slice/crio-f87a6b3e925ac4231b322179eb9b9b5af2061531280cefe71483e1ae6eb37ba1 WatchSource:0}: Error finding container f87a6b3e925ac4231b322179eb9b9b5af2061531280cefe71483e1ae6eb37ba1: Status 404 returned error can't find the container with id f87a6b3e925ac4231b322179eb9b9b5af2061531280cefe71483e1ae6eb37ba1
Jul 10 20:42:28.904620 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.904519    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ts8f9"]
Jul 10 20:42:28.907351 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:28.907305    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2265ffde_452f_4d13_9350_f0ba6bc280a2.slice/crio-3395075c9ba0eb84483a5fc32d7f752a22f76dc6ee106e02008f92c51a06e0da WatchSource:0}: Error finding container 3395075c9ba0eb84483a5fc32d7f752a22f76dc6ee106e02008f92c51a06e0da: Status 404 returned error can't find the container with id 3395075c9ba0eb84483a5fc32d7f752a22f76dc6ee106e02008f92c51a06e0da
Jul 10 20:42:28.931008 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:28.930976    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bb489dc77-mfmzt"]
Jul 10 20:42:28.933428 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:28.933401    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e3380a_9607_4464_879c_f4bac8dff3c0.slice/crio-e13d0d67b9075a03214ee5069a9097ea970b55edeed99394c2cf9e17ed43e8c6 WatchSource:0}: Error finding container e13d0d67b9075a03214ee5069a9097ea970b55edeed99394c2cf9e17ed43e8c6: Status 404 returned error can't find the container with id e13d0d67b9075a03214ee5069a9097ea970b55edeed99394c2cf9e17ed43e8c6
Jul 10 20:42:29.851453 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:29.851413    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt" event={"ID":"48e3380a-9607-4464-879c-f4bac8dff3c0","Type":"ContainerStarted","Data":"3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b"}
Jul 10 20:42:29.851912 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:29.851475    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt" event={"ID":"48e3380a-9607-4464-879c-f4bac8dff3c0","Type":"ContainerStarted","Data":"e13d0d67b9075a03214ee5069a9097ea970b55edeed99394c2cf9e17ed43e8c6"}
Jul 10 20:42:29.851912 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:29.851503    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:42:29.852929 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:29.852874    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk" event={"ID":"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d","Type":"ContainerStarted","Data":"f87a6b3e925ac4231b322179eb9b9b5af2061531280cefe71483e1ae6eb37ba1"}
Jul 10 20:42:29.854199 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:29.854158    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ts8f9" event={"ID":"2265ffde-452f-4d13-9350-f0ba6bc280a2","Type":"ContainerStarted","Data":"3395075c9ba0eb84483a5fc32d7f752a22f76dc6ee106e02008f92c51a06e0da"}
Jul 10 20:42:29.871574 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:29.871509    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt" podStartSLOduration=285.8714914 podStartE2EDuration="4m45.8714914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:29.870729795 +0000 UTC m=+286.363525999" watchObservedRunningTime="2026-07-10 20:42:29.8714914 +0000 UTC m=+286.364287575"
Jul 10 20:42:30.860270 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:30.860238    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk" event={"ID":"eedc1267-4e6d-4dcf-b1eb-1232f2214f2d","Type":"ContainerStarted","Data":"2d18647abf990e6a8f63296bf47428a1b1c34284d60cd5b34ce1f717c6b8b7dd"}
Jul 10 20:42:30.861894 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:30.861864    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ts8f9" event={"ID":"2265ffde-452f-4d13-9350-f0ba6bc280a2","Type":"ContainerStarted","Data":"5f621105dd3e0ba282b4f33f5121a384fd8809b3a1a17615d5ef36be09d33cbf"}
Jul 10 20:42:30.876547 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:30.876503    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-wcxlk" podStartSLOduration=283.733650294 podStartE2EDuration="4m44.876483485s" totalImagesPullingTime="1.142833191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:30.874657334 +0000 UTC m=+287.367453514" watchObservedRunningTime="2026-07-10 20:42:30.876483485 +0000 UTC m=+287.369279664"
Jul 10 20:42:30.963285 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:30.963185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:30.965523 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:30.965499    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/50d32382-94d8-407b-aeef-4b0490d751bf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgsdg\" (UID: \"50d32382-94d8-407b-aeef-4b0490d751bf\") " pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:31.159743 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.159698    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pgsdg"
Jul 10 20:42:31.302488 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.302457    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pgsdg"]
Jul 10 20:42:31.305494 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:31.305466    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d32382_94d8_407b_aeef_4b0490d751bf.slice/crio-9a68b36af061fb8c3854315bfb9f794983200164084d073b3b843d4019d1f8a1 WatchSource:0}: Error finding container 9a68b36af061fb8c3854315bfb9f794983200164084d073b3b843d4019d1f8a1: Status 404 returned error can't find the container with id 9a68b36af061fb8c3854315bfb9f794983200164084d073b3b843d4019d1f8a1
Jul 10 20:42:31.532590 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.532495    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-2gjnt"]
Jul 10 20:42:31.560918 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.560892    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-2gjnt"]
Jul 10 20:42:31.561052 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.560998    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.565691 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.565629    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 10 20:42:31.565691 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.565677    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 20:42:31.565691 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.565689    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-q47z4\""
Jul 10 20:42:31.565931 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.565706    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 10 20:42:31.565931 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.565718    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 20:42:31.565931 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.565928    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 20:42:31.669593 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.669538    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hnt2\" (UniqueName: \"kubernetes.io/projected/c184f503-61b0-4230-84e2-3ff6b2a19aa9-kube-api-access-4hnt2\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.669593 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.669587    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c184f503-61b0-4230-84e2-3ff6b2a19aa9-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.669783 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.669655    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c184f503-61b0-4230-84e2-3ff6b2a19aa9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.669783 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.669684    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c184f503-61b0-4230-84e2-3ff6b2a19aa9-metrics-client-ca\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.770646 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.770604    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c184f503-61b0-4230-84e2-3ff6b2a19aa9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.770646 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.770656    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c184f503-61b0-4230-84e2-3ff6b2a19aa9-metrics-client-ca\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.770902 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.770752    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hnt2\" (UniqueName: \"kubernetes.io/projected/c184f503-61b0-4230-84e2-3ff6b2a19aa9-kube-api-access-4hnt2\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.770902 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.770780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c184f503-61b0-4230-84e2-3ff6b2a19aa9-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.771525 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.771479    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c184f503-61b0-4230-84e2-3ff6b2a19aa9-metrics-client-ca\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.773538 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.773509    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c184f503-61b0-4230-84e2-3ff6b2a19aa9-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.773664 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.773602    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c184f503-61b0-4230-84e2-3ff6b2a19aa9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.779396 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.779371    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hnt2\" (UniqueName: \"kubernetes.io/projected/c184f503-61b0-4230-84e2-3ff6b2a19aa9-kube-api-access-4hnt2\") pod \"prometheus-operator-64785b6675-2gjnt\" (UID: \"c184f503-61b0-4230-84e2-3ff6b2a19aa9\") " pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.868698 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.868658    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ts8f9" event={"ID":"2265ffde-452f-4d13-9350-f0ba6bc280a2","Type":"ContainerStarted","Data":"8c3fcc98a46634d7f868b0b1834eb15ed3e7520bdd9d6a23dbe29782422f710d"}
Jul 10 20:42:31.869131 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.868842    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:42:31.869680 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.869655    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt"
Jul 10 20:42:31.870673 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.870622    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgsdg" event={"ID":"50d32382-94d8-407b-aeef-4b0490d751bf","Type":"ContainerStarted","Data":"9a68b36af061fb8c3854315bfb9f794983200164084d073b3b843d4019d1f8a1"}
Jul 10 20:42:31.886995 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:31.886805    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ts8f9" podStartSLOduration=252.086536803 podStartE2EDuration="4m13.886784895s" totalImagesPullingTime="1.800248092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:31.884882902 +0000 UTC m=+288.377679083" watchObservedRunningTime="2026-07-10 20:42:31.886784895 +0000 UTC m=+288.379581074"
Jul 10 20:42:32.044500 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:32.044415    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-2gjnt"]
Jul 10 20:42:32.049831 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:32.049768    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc184f503_61b0_4230_84e2_3ff6b2a19aa9.slice/crio-16f2a08b65b53e03e6aa302d1af4ae24313469f921f1ee71ec88b2a0e9124028 WatchSource:0}: Error finding container 16f2a08b65b53e03e6aa302d1af4ae24313469f921f1ee71ec88b2a0e9124028: Status 404 returned error can't find the container with id 16f2a08b65b53e03e6aa302d1af4ae24313469f921f1ee71ec88b2a0e9124028
Jul 10 20:42:32.874602 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:32.874549    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt" event={"ID":"c184f503-61b0-4230-84e2-3ff6b2a19aa9","Type":"ContainerStarted","Data":"16f2a08b65b53e03e6aa302d1af4ae24313469f921f1ee71ec88b2a0e9124028"}
Jul 10 20:42:35.887548 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:35.887511    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgsdg" event={"ID":"50d32382-94d8-407b-aeef-4b0490d751bf","Type":"ContainerStarted","Data":"af70fb8d389360dadee68503bd4800eb07e94f56668d52322e6d9131350f1d23"}
Jul 10 20:42:35.889274 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:35.889246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt" event={"ID":"c184f503-61b0-4230-84e2-3ff6b2a19aa9","Type":"ContainerStarted","Data":"f03ee0fa6c90074e9a275fae7ba557d98c04d3f43fd335414405f0a14a68196f"}
Jul 10 20:42:35.889383 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:35.889300    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt" event={"ID":"c184f503-61b0-4230-84e2-3ff6b2a19aa9","Type":"ContainerStarted","Data":"39d3bb593471fbb52d99b4e2020405f546bfe22ab14645084bd5f2cccdd5cec7"}
Jul 10 20:42:35.907441 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:35.907401    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-2gjnt" podStartSLOduration=1.489784629 podStartE2EDuration="4.907387159s" totalImagesPullingTime="3.41760253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:35.906081624 +0000 UTC m=+292.398877803" watchObservedRunningTime="2026-07-10 20:42:35.907387159 +0000 UTC m=+292.400183337"
Jul 10 20:42:36.951505 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:36.951483    2565 scope.go:117] "RemoveContainer" containerID="b36a29b165e1d9a0336701e828e3ba8223927b7354c0c9e53c86abce88b522d0"
Jul 10 20:42:36.951778 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:36.951689    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-zxhjm_openshift-console-operator(cc40c181-ad1c-4ef7-91cf-b35d8343f590)\"" pod="openshift-console-operator/console-operator-7866497494-zxhjm" podUID="cc40c181-ad1c-4ef7-91cf-b35d8343f590"
Jul 10 20:42:37.896483 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:37.896445    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgsdg" event={"ID":"50d32382-94d8-407b-aeef-4b0490d751bf","Type":"ContainerStarted","Data":"9347cdc990cce967c278eb096d9bac04c4dadf025c3ee9fbb8df71e08528fa25"}
Jul 10 20:42:37.951845 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:37.951807    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nbqvl" podUID="5436825a-b217-4c2d-a358-97173d8da40a"
Jul 10 20:42:37.958844 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:37.958817    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dfncg"]
Jul 10 20:42:37.962179 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:37.962158    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:37.964903 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:37.964882    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 20:42:37.965081 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:37.965062    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 20:42:37.965215 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:37.965201    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 20:42:37.965276 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:37.965232    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-chckk\""
Jul 10 20:42:38.027131 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.027096    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r92ql\" (UniqueName: \"kubernetes.io/projected/265baed6-9e3d-42d0-9250-a4129f1ee5d8-kube-api-access-r92ql\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.027292 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.027162    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/265baed6-9e3d-42d0-9250-a4129f1ee5d8-sys\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.027292 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.027200    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.027292 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.027224    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/265baed6-9e3d-42d0-9250-a4129f1ee5d8-root\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.027292 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.027268    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-tls\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.027466 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.027315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265baed6-9e3d-42d0-9250-a4129f1ee5d8-metrics-client-ca\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.027466 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.027371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.027466 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.027393    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-textfile\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.027466 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.027409    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-wtmp\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128261 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128221    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r92ql\" (UniqueName: \"kubernetes.io/projected/265baed6-9e3d-42d0-9250-a4129f1ee5d8-kube-api-access-r92ql\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128484 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128299    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/265baed6-9e3d-42d0-9250-a4129f1ee5d8-sys\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128484 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128340    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128484 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128369    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/265baed6-9e3d-42d0-9250-a4129f1ee5d8-root\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128484 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128393    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-tls\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128484 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128414    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265baed6-9e3d-42d0-9250-a4129f1ee5d8-metrics-client-ca\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128484 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128446    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128484 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128474    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-textfile\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128495    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-wtmp\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128836    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/265baed6-9e3d-42d0-9250-a4129f1ee5d8-sys\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/265baed6-9e3d-42d0-9250-a4129f1ee5d8-root\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.128917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.128897    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-textfile\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.129106 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.129068    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-wtmp\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.129236 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.129208    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265baed6-9e3d-42d0-9250-a4129f1ee5d8-metrics-client-ca\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.129397 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.129298    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.131660 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.131615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.132541 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.132515    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/265baed6-9e3d-42d0-9250-a4129f1ee5d8-node-exporter-tls\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.139431 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.139381    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r92ql\" (UniqueName: \"kubernetes.io/projected/265baed6-9e3d-42d0-9250-a4129f1ee5d8-kube-api-access-r92ql\") pod \"node-exporter-dfncg\" (UID: \"265baed6-9e3d-42d0-9250-a4129f1ee5d8\") " pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.270825 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.270743    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dfncg"
Jul 10 20:42:38.283348 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:38.283310    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265baed6_9e3d_42d0_9250_a4129f1ee5d8.slice/crio-3ce71188dd8948168111efb0dbcee9335f95f064376fe82421cb046a992953ce WatchSource:0}: Error finding container 3ce71188dd8948168111efb0dbcee9335f95f064376fe82421cb046a992953ce: Status 404 returned error can't find the container with id 3ce71188dd8948168111efb0dbcee9335f95f064376fe82421cb046a992953ce
Jul 10 20:42:38.902832 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.902790    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dfncg" event={"ID":"265baed6-9e3d-42d0-9250-a4129f1ee5d8","Type":"ContainerStarted","Data":"3ce71188dd8948168111efb0dbcee9335f95f064376fe82421cb046a992953ce"}
Jul 10 20:42:38.998477 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:38.998240    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:42:39.003883 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.003855    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.006637 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.006611    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 20:42:39.006776 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.006649    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 20:42:39.006776 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.006679    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 20:42:39.006776 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.006621    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 20:42:39.006942 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.006880    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 20:42:39.006942 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.006926    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 20:42:39.007038 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.006950    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7chfv\""
Jul 10 20:42:39.007179 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.007160    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 20:42:39.007230 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.007162    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 20:42:39.007230 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.007166    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 20:42:39.019608 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.019499    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:42:39.138045 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.137997    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138217 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138113    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl895\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-kube-api-access-nl895\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138217 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138161    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138217 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138201    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138377 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138267    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138377 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138303    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138377 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138330    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-web-config\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138377 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138357    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-config-volume\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138591 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138414    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138591 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138432    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138591 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138457    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138591 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138509    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-config-out\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.138591 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.138541    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239230 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239136    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239230 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239209    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239454 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239249    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239454 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239269    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239454 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239293    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-web-config\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239454 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239319    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-config-volume\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239454 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239355    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239454 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239379    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239454 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239410    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239454 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:39.239434    2565 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 10 20:42:39.239875 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-config-out\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239875 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239486    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239875 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:39.239504    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-main-tls podName:44af0481-a822-4861-9199-cbb71e604a5c nodeName:}" failed. No retries permitted until 2026-07-10 20:42:39.739483981 +0000 UTC m=+296.232280148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "44af0481-a822-4861-9199-cbb71e604a5c") : secret "alertmanager-main-tls" not found
Jul 10 20:42:39.239875 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.239875 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.239631    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl895\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-kube-api-access-nl895\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.240732 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.240386    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.240875 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.240776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.240934 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.240881    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.242666 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.242620    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.242844 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.242798    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.243166 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.243143    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-config-out\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.243516 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.243494    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.243620 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.243517    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.244504 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.244462    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-config-volume\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.244594 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.244512    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.244664 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.244644    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-web-config\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.250620 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.250595    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl895\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-kube-api-access-nl895\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.746133 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.746083    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.748953 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.748917    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:39.891830 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.891795    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"]
Jul 10 20:42:39.903573 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.903514    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:39.907932 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.907361    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"]
Jul 10 20:42:39.907932 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.907836    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 10 20:42:39.907932 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.907891    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 10 20:42:39.908193 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.908173    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jqb27\""
Jul 10 20:42:39.908263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.908246    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cdd77v6u0ccsv\""
Jul 10 20:42:39.908602 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.908433    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 10 20:42:39.908602 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.908460    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 10 20:42:39.908602 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.908470    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 10 20:42:39.916307 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:39.916288    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:42:40.048760 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.048653    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-metrics-client-ca\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.048760 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.048719    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-tls\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.049178 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.048812    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-grpc-tls\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.049178 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.048866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.049178 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.048939    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52899\" (UniqueName: \"kubernetes.io/projected/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-kube-api-access-52899\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.049178 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.048983    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.049178 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.049034    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.049178 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.049076    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.150352 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.150309    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.150534 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.150373    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.150534 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.150431    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-metrics-client-ca\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.150534 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.150492    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-tls\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.150534 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.150535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-grpc-tls\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.150776 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.150603    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.150776 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.150648    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52899\" (UniqueName: \"kubernetes.io/projected/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-kube-api-access-52899\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.150776 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.150678    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.151312 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.151281    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-metrics-client-ca\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.153706 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.153611    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.153706 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.153654    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.153873 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.153847    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.154054 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.154034    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.154207 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.154173    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-thanos-querier-tls\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.154527 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.154506    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-secret-grpc-tls\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.159169 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.159144    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52899\" (UniqueName: \"kubernetes.io/projected/2d43a49a-a3ef-4309-8bb0-f23362fa73aa-kube-api-access-52899\") pod \"thanos-querier-5cbf9ddd68-4j7kn\" (UID: \"2d43a49a-a3ef-4309-8bb0-f23362fa73aa\") " pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.216015 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.215979    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:40.464883 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.464788    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"]
Jul 10 20:42:40.467500 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:40.467469    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d43a49a_a3ef_4309_8bb0_f23362fa73aa.slice/crio-6ab58a31c9d7a3c51e145ce838f0fc80de49206760fd422717860910d4578552 WatchSource:0}: Error finding container 6ab58a31c9d7a3c51e145ce838f0fc80de49206760fd422717860910d4578552: Status 404 returned error can't find the container with id 6ab58a31c9d7a3c51e145ce838f0fc80de49206760fd422717860910d4578552
Jul 10 20:42:40.488445 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.488417    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:42:40.538379 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:40.538345    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44af0481_a822_4861_9199_cbb71e604a5c.slice/crio-8f08889c76b3c20ebdaf5997e6ecf370f277e8af0af80f2494c675218d49abfa WatchSource:0}: Error finding container 8f08889c76b3c20ebdaf5997e6ecf370f277e8af0af80f2494c675218d49abfa: Status 404 returned error can't find the container with id 8f08889c76b3c20ebdaf5997e6ecf370f277e8af0af80f2494c675218d49abfa
Jul 10 20:42:40.915308 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.915272    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgsdg" event={"ID":"50d32382-94d8-407b-aeef-4b0490d751bf","Type":"ContainerStarted","Data":"10b24cad525cf9226e9f832444c200c5f6b3bc7266c4de500267a7b8f72e4d8b"}
Jul 10 20:42:40.916841 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.916815    2565 generic.go:356] "Generic (PLEG): container finished" podID="265baed6-9e3d-42d0-9250-a4129f1ee5d8" containerID="e28f9404674a21980e752b798f8d1e783800617ec97a4ec8d2c7bd25493712c5" exitCode=0
Jul 10 20:42:40.916953 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.916884    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dfncg" event={"ID":"265baed6-9e3d-42d0-9250-a4129f1ee5d8","Type":"ContainerDied","Data":"e28f9404674a21980e752b798f8d1e783800617ec97a4ec8d2c7bd25493712c5"}
Jul 10 20:42:40.918040 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.918011    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn" event={"ID":"2d43a49a-a3ef-4309-8bb0-f23362fa73aa","Type":"ContainerStarted","Data":"6ab58a31c9d7a3c51e145ce838f0fc80de49206760fd422717860910d4578552"}
Jul 10 20:42:40.919166 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.919119    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerStarted","Data":"8f08889c76b3c20ebdaf5997e6ecf370f277e8af0af80f2494c675218d49abfa"}
Jul 10 20:42:40.932320 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:40.932276    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pgsdg" podStartSLOduration=33.072431863 podStartE2EDuration="41.932260557s" totalImagesPullingTime="8.859828694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:41:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:40.931658092 +0000 UTC m=+297.424454272" watchObservedRunningTime="2026-07-10 20:42:40.932260557 +0000 UTC m=+297.425056735"
Jul 10 20:42:41.876858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:41.876831    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ts8f9"
Jul 10 20:42:41.925071 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:41.925030    2565 generic.go:356] "Generic (PLEG): container finished" podID="44af0481-a822-4861-9199-cbb71e604a5c" containerID="e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b" exitCode=0
Jul 10 20:42:41.925369 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:41.925326    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerDied","Data":"e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b"}
Jul 10 20:42:41.929436 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:41.928627    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dfncg" event={"ID":"265baed6-9e3d-42d0-9250-a4129f1ee5d8","Type":"ContainerStarted","Data":"0ba0fb7d08a926ef33f74bbcfea6c3239c8c6689ef322981d0363cf8514ec4fa"}
Jul 10 20:42:41.929436 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:41.928683    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dfncg" event={"ID":"265baed6-9e3d-42d0-9250-a4129f1ee5d8","Type":"ContainerStarted","Data":"44774da75f5e3871d9daacee4172992d35c01fce3875fa6d25605e4f711a68b7"}
Jul 10 20:42:41.965872 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:41.965827    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dfncg" podStartSLOduration=2.856320138 podStartE2EDuration="4.965811596s" totalImagesPullingTime="2.03077416s" totalInitContainerRuntime="78.717298ms" isStatefulPod=true podCreationTimestamp="2026-07-10 20:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:41.964637841 +0000 UTC m=+298.457434019" watchObservedRunningTime="2026-07-10 20:42:41.965811596 +0000 UTC m=+298.458607774"
Jul 10 20:42:42.671394 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.671359    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"]
Jul 10 20:42:42.675915 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.675888    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:42.679842 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.679808    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 10 20:42:42.679842 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.679808    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wjn45\""
Jul 10 20:42:42.683370 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.683343    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"]
Jul 10 20:42:42.780888 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.780843    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwtpb\" (UniqueName: \"kubernetes.io/projected/77245f92-3a87-45fc-8024-64ab9843d1e0-kube-api-access-bwtpb\") pod \"monitoring-plugin-5f68df7f58-tz5ck\" (UID: \"77245f92-3a87-45fc-8024-64ab9843d1e0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:42.781008 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.780973    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77245f92-3a87-45fc-8024-64ab9843d1e0-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tz5ck\" (UID: \"77245f92-3a87-45fc-8024-64ab9843d1e0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:42.882739 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.882665    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwtpb\" (UniqueName: \"kubernetes.io/projected/77245f92-3a87-45fc-8024-64ab9843d1e0-kube-api-access-bwtpb\") pod \"monitoring-plugin-5f68df7f58-tz5ck\" (UID: \"77245f92-3a87-45fc-8024-64ab9843d1e0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:42.883063 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.882746    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77245f92-3a87-45fc-8024-64ab9843d1e0-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tz5ck\" (UID: \"77245f92-3a87-45fc-8024-64ab9843d1e0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:42.883063 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:42.882844    2565 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 10 20:42:42.883063 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:42:42.882898    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77245f92-3a87-45fc-8024-64ab9843d1e0-monitoring-plugin-cert podName:77245f92-3a87-45fc-8024-64ab9843d1e0 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:43.382883433 +0000 UTC m=+299.875679590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/77245f92-3a87-45fc-8024-64ab9843d1e0-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-tz5ck" (UID: "77245f92-3a87-45fc-8024-64ab9843d1e0") : secret "monitoring-plugin-cert" not found
Jul 10 20:42:42.893666 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.893631    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwtpb\" (UniqueName: \"kubernetes.io/projected/77245f92-3a87-45fc-8024-64ab9843d1e0-kube-api-access-bwtpb\") pod \"monitoring-plugin-5f68df7f58-tz5ck\" (UID: \"77245f92-3a87-45fc-8024-64ab9843d1e0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:42.934797 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.934758    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn" event={"ID":"2d43a49a-a3ef-4309-8bb0-f23362fa73aa","Type":"ContainerStarted","Data":"4f03958be949f6d96c4160d9f845d62b17f95d2e53933033aeb455c1dcbda1c6"}
Jul 10 20:42:42.934969 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.934858    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn" event={"ID":"2d43a49a-a3ef-4309-8bb0-f23362fa73aa","Type":"ContainerStarted","Data":"cae94d275ee9fd54b655705c5ae6a91f88850b3579bc8405b4099ec01627a54e"}
Jul 10 20:42:42.934969 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:42.934885    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn" event={"ID":"2d43a49a-a3ef-4309-8bb0-f23362fa73aa","Type":"ContainerStarted","Data":"c4f3c3376975f663f6212e6ae3acad7086e5298131e33e1daa052272fdf4cfdd"}
Jul 10 20:42:43.387542 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:43.387496    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77245f92-3a87-45fc-8024-64ab9843d1e0-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tz5ck\" (UID: \"77245f92-3a87-45fc-8024-64ab9843d1e0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:43.390685 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:43.390649    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77245f92-3a87-45fc-8024-64ab9843d1e0-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tz5ck\" (UID: \"77245f92-3a87-45fc-8024-64ab9843d1e0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:43.592332 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:43.592299    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:44.033908 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.033769    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bb489dc77-mfmzt"]
Jul 10 20:42:44.046504 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.038544    2565 patch_prober.go:28] interesting pod/image-registry-bb489dc77-mfmzt container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 20:42:44.046504 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.038626    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt" podUID="48e3380a-9607-4464-879c-f4bac8dff3c0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 20:42:44.082516 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.082488    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:42:44.083857 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.083833    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:42:44.104549 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.102742    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:42:44.104549 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.104075    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:42:44.110773 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.110748    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 20:42:44.236422 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.236288    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"]
Jul 10 20:42:44.240610 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:44.240575    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77245f92_3a87_45fc_8024_64ab9843d1e0.slice/crio-9c30e4c2500281b4a2fc6dda69aee905f99d92761a63f595335dd858452305bc WatchSource:0}: Error finding container 9c30e4c2500281b4a2fc6dda69aee905f99d92761a63f595335dd858452305bc: Status 404 returned error can't find the container with id 9c30e4c2500281b4a2fc6dda69aee905f99d92761a63f595335dd858452305bc
Jul 10 20:42:44.944211 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.944172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck" event={"ID":"77245f92-3a87-45fc-8024-64ab9843d1e0","Type":"ContainerStarted","Data":"9c30e4c2500281b4a2fc6dda69aee905f99d92761a63f595335dd858452305bc"}
Jul 10 20:42:44.947301 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.947260    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn" event={"ID":"2d43a49a-a3ef-4309-8bb0-f23362fa73aa","Type":"ContainerStarted","Data":"d79a27b3d55cdc45eb020a0fee95c83f615c40e59649f5cf000c8be2baffeb76"}
Jul 10 20:42:44.947433 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.947327    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn" event={"ID":"2d43a49a-a3ef-4309-8bb0-f23362fa73aa","Type":"ContainerStarted","Data":"283cda585a9d9220559d9656954d2dbfe5de550848158231bfc3cc43f50325b9"}
Jul 10 20:42:44.947433 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.947364    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn" event={"ID":"2d43a49a-a3ef-4309-8bb0-f23362fa73aa","Type":"ContainerStarted","Data":"2422d7989da3d0cfe3c05713726bf3e79eeacc974ea1098ee45b6bd4448f0114"}
Jul 10 20:42:44.947433 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.947407    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:44.951232 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.951205    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerStarted","Data":"62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400"}
Jul 10 20:42:44.951376 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.951274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerStarted","Data":"1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5"}
Jul 10 20:42:44.951376 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.951315    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerStarted","Data":"bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9"}
Jul 10 20:42:44.951376 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.951352    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerStarted","Data":"e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45"}
Jul 10 20:42:44.951481 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.951401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerStarted","Data":"7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f"}
Jul 10 20:42:44.951481 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.951431    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerStarted","Data":"1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9"}
Jul 10 20:42:44.971894 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.971841    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn" podStartSLOduration=2.618799076 podStartE2EDuration="5.971821283s" totalImagesPullingTime="3.353022207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:44.969912322 +0000 UTC m=+301.462708504" watchObservedRunningTime="2026-07-10 20:42:44.971821283 +0000 UTC m=+301.464617463"
Jul 10 20:42:44.996581 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:44.996498    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.630395528 podStartE2EDuration="6.996482138s" totalImagesPullingTime="3.350106321s" totalInitContainerRuntime="15.980289ms" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:44.993962466 +0000 UTC m=+301.486758645" watchObservedRunningTime="2026-07-10 20:42:44.996482138 +0000 UTC m=+301.489278317"
Jul 10 20:42:45.959535 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:45.959496    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck" event={"ID":"77245f92-3a87-45fc-8024-64ab9843d1e0","Type":"ContainerStarted","Data":"d0bfaed1b5df218bcc2f560a0525179cf423875ccc1cd01532737595f2955235"}
Jul 10 20:42:45.960141 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:45.960117    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:45.964793 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:45.964773    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck"
Jul 10 20:42:45.975367 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:45.975289    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tz5ck" podStartSLOduration=2.5006606749999998 podStartE2EDuration="3.975276086s" totalImagesPullingTime="1.474615411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:45.974595762 +0000 UTC m=+302.467391940" watchObservedRunningTime="2026-07-10 20:42:45.975276086 +0000 UTC m=+302.468072264"
Jul 10 20:42:48.951427 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:48.951389    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:42:50.965977 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:50.965944    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5cbf9ddd68-4j7kn"
Jul 10 20:42:51.951519 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:51.951479    2565 scope.go:117] "RemoveContainer" containerID="b36a29b165e1d9a0336701e828e3ba8223927b7354c0c9e53c86abce88b522d0"
Jul 10 20:42:52.988232 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:52.988205    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:42:52.988640 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:52.988269    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-zxhjm" event={"ID":"cc40c181-ad1c-4ef7-91cf-b35d8343f590","Type":"ContainerStarted","Data":"34832dca3d8f838c44dc6ad3820ce087ab1c0219604e49d816081fda94d64ee9"}
Jul 10 20:42:52.988640 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:52.988572    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:42:53.007886 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.007835    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-zxhjm" podStartSLOduration=55.772630319 podStartE2EDuration="59.007820833s" totalImagesPullingTime="3.235190514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:53.00618809 +0000 UTC m=+309.498984271" watchObservedRunningTime="2026-07-10 20:42:53.007820833 +0000 UTC m=+309.500617011"
Jul 10 20:42:53.205343 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.205312    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-zxhjm"
Jul 10 20:42:53.379507 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.379467    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-p9l79"]
Jul 10 20:42:53.382619 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.382597    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-p9l79"
Jul 10 20:42:53.385223 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.385198    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rb76p\""
Jul 10 20:42:53.385304 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.385202    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 20:42:53.385304 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.385206    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 20:42:53.391660 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.391638    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-p9l79"]
Jul 10 20:42:53.485842 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.485806    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqx48\" (UniqueName: \"kubernetes.io/projected/bb033ea7-58e3-46e6-93b5-ed1c1b11c632-kube-api-access-kqx48\") pod \"downloads-68c9cbb497-p9l79\" (UID: \"bb033ea7-58e3-46e6-93b5-ed1c1b11c632\") " pod="openshift-console/downloads-68c9cbb497-p9l79"
Jul 10 20:42:53.586595 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.586535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqx48\" (UniqueName: \"kubernetes.io/projected/bb033ea7-58e3-46e6-93b5-ed1c1b11c632-kube-api-access-kqx48\") pod \"downloads-68c9cbb497-p9l79\" (UID: \"bb033ea7-58e3-46e6-93b5-ed1c1b11c632\") " pod="openshift-console/downloads-68c9cbb497-p9l79"
Jul 10 20:42:53.595208 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.595185    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqx48\" (UniqueName: \"kubernetes.io/projected/bb033ea7-58e3-46e6-93b5-ed1c1b11c632-kube-api-access-kqx48\") pod \"downloads-68c9cbb497-p9l79\" (UID: \"bb033ea7-58e3-46e6-93b5-ed1c1b11c632\") " pod="openshift-console/downloads-68c9cbb497-p9l79"
Jul 10 20:42:53.692920 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.692839    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-p9l79"
Jul 10 20:42:53.819727 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.819700    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-p9l79"]
Jul 10 20:42:53.822740 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:53.822700    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb033ea7_58e3_46e6_93b5_ed1c1b11c632.slice/crio-252795ca98bb9ad9598359a3b7b4ad67c26663b8c0f5689b191b6c50c56a4e6e WatchSource:0}: Error finding container 252795ca98bb9ad9598359a3b7b4ad67c26663b8c0f5689b191b6c50c56a4e6e: Status 404 returned error can't find the container with id 252795ca98bb9ad9598359a3b7b4ad67c26663b8c0f5689b191b6c50c56a4e6e
Jul 10 20:42:53.824822 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.824802    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 20:42:53.992391 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:53.992315    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-p9l79" event={"ID":"bb033ea7-58e3-46e6-93b5-ed1c1b11c632","Type":"ContainerStarted","Data":"252795ca98bb9ad9598359a3b7b4ad67c26663b8c0f5689b191b6c50c56a4e6e"}
Jul 10 20:42:54.038117 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:54.038083    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:42:59.345075 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.345036    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cddb8b78c-j5wn6"]
Jul 10 20:42:59.348970 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.348947    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.353198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.352949    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 20:42:59.353198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.353007    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h8pjl\""
Jul 10 20:42:59.353198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.353007    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 20:42:59.353198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.353074    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 20:42:59.353198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.353029    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 20:42:59.353198 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.353164    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 20:42:59.361074 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.361052    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cddb8b78c-j5wn6"]
Jul 10 20:42:59.436244 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.436193    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-oauth-serving-cert\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.436244 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.436241    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-service-ca\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.436471 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.436336    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-oauth-config\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.436471 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.436394    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-console-config\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.436653 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.436467    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v75dr\" (UniqueName: \"kubernetes.io/projected/484b9d14-48c3-40b7-91ab-d546b262faf5-kube-api-access-v75dr\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.436653 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.436532    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-serving-cert\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.537569 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.537520    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-console-config\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.537764 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.537581    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v75dr\" (UniqueName: \"kubernetes.io/projected/484b9d14-48c3-40b7-91ab-d546b262faf5-kube-api-access-v75dr\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.537764 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.537628    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-serving-cert\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.537764 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.537652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-oauth-serving-cert\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.537764 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.537669    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-service-ca\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.537764 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.537704    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-oauth-config\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.538419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.538379    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-oauth-serving-cert\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.538419 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.538380    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-console-config\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.538637 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.538585    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-service-ca\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.540996 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.540972    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-oauth-config\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.541420 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.541401    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-serving-cert\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.546698 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.546656    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v75dr\" (UniqueName: \"kubernetes.io/projected/484b9d14-48c3-40b7-91ab-d546b262faf5-kube-api-access-v75dr\") pod \"console-7cddb8b78c-j5wn6\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") " pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.661782 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.661697    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:42:59.803110 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:42:59.803064    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cddb8b78c-j5wn6"]
Jul 10 20:42:59.806103 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:42:59.806076    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484b9d14_48c3_40b7_91ab_d546b262faf5.slice/crio-54bf9fd673bbe10d6a5831d47ce34828004d778f08e40a64b1448646bbdca8af WatchSource:0}: Error finding container 54bf9fd673bbe10d6a5831d47ce34828004d778f08e40a64b1448646bbdca8af: Status 404 returned error can't find the container with id 54bf9fd673bbe10d6a5831d47ce34828004d778f08e40a64b1448646bbdca8af
Jul 10 20:43:00.019949 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:00.019863    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cddb8b78c-j5wn6" event={"ID":"484b9d14-48c3-40b7-91ab-d546b262faf5","Type":"ContainerStarted","Data":"54bf9fd673bbe10d6a5831d47ce34828004d778f08e40a64b1448646bbdca8af"}
Jul 10 20:43:04.042798 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:04.042753    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cddb8b78c-j5wn6" event={"ID":"484b9d14-48c3-40b7-91ab-d546b262faf5","Type":"ContainerStarted","Data":"d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc"}
Jul 10 20:43:04.062786 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:04.062731    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cddb8b78c-j5wn6" podStartSLOduration=1.682104416 podStartE2EDuration="5.062709862s" totalImagesPullingTime="3.380605446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:04.060031251 +0000 UTC m=+320.552827434" watchObservedRunningTime="2026-07-10 20:43:04.062709862 +0000 UTC m=+320.555506042"
Jul 10 20:43:09.059954 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.059905    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt" podUID="48e3380a-9607-4464-879c-f4bac8dff3c0" containerName="registry" containerID="cri-o://3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b" gracePeriod=30
Jul 10 20:43:09.662130 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.662091    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:43:09.662311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.662183    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:43:09.667922 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.667898    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:43:09.683847 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.683803    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bccd78f9d-5rzpm"]
Jul 10 20:43:09.687539 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.687514    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.694951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.694926    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 10 20:43:09.698332 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.698309    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bccd78f9d-5rzpm"]
Jul 10 20:43:09.737330 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.737301    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-oauth-config\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.737478 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.737335    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-oauth-serving-cert\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.737478 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.737424    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-service-ca\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.737648 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.737606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-serving-cert\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.737734 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.737712    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-console-config\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.737787 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.737778    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-trusted-ca-bundle\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.737845 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.737806    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tjx8\" (UniqueName: \"kubernetes.io/projected/5153ec06-6263-4270-9d0d-357307925669-kube-api-access-4tjx8\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.838493 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.838451    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-oauth-serving-cert\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.838706 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.838524    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-service-ca\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.838706 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.838608    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-serving-cert\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.838706 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.838660    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-console-config\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.838706 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.838694    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-trusted-ca-bundle\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.838917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.838717    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tjx8\" (UniqueName: \"kubernetes.io/projected/5153ec06-6263-4270-9d0d-357307925669-kube-api-access-4tjx8\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.838917 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.838774    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-oauth-config\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.839290 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.839243    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-oauth-serving-cert\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.839404 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.839314    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-service-ca\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.839667 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.839641    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-console-config\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.840057 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.840032    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-trusted-ca-bundle\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.841625 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.841604    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-serving-cert\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.841721 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.841702    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-oauth-config\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.847484 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.847461    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tjx8\" (UniqueName: \"kubernetes.io/projected/5153ec06-6263-4270-9d0d-357307925669-kube-api-access-4tjx8\") pod \"console-7bccd78f9d-5rzpm\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") " pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:09.999832 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:09.999755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:10.076317 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.076288    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:43:10.754634 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.754609    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bccd78f9d-5rzpm"]
Jul 10 20:43:10.755405 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.755382    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:43:10.762591 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:43:10.762530    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5153ec06_6263_4270_9d0d_357307925669.slice/crio-e2d4fa21f2fa51dcc4eb333c4c30c508dad146b9e6922a6773599d81b0e53190 WatchSource:0}: Error finding container e2d4fa21f2fa51dcc4eb333c4c30c508dad146b9e6922a6773599d81b0e53190: Status 404 returned error can't find the container with id e2d4fa21f2fa51dcc4eb333c4c30c508dad146b9e6922a6773599d81b0e53190
Jul 10 20:43:10.847049 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847012    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-trusted-ca\") pod \"48e3380a-9607-4464-879c-f4bac8dff3c0\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") "
Jul 10 20:43:10.847195 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847083    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-bound-sa-token\") pod \"48e3380a-9607-4464-879c-f4bac8dff3c0\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") "
Jul 10 20:43:10.847195 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847126    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-certificates\") pod \"48e3380a-9607-4464-879c-f4bac8dff3c0\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") "
Jul 10 20:43:10.847195 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847183    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e3380a-9607-4464-879c-f4bac8dff3c0-ca-trust-extracted\") pod \"48e3380a-9607-4464-879c-f4bac8dff3c0\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") "
Jul 10 20:43:10.847351 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847217    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-image-registry-private-configuration\") pod \"48e3380a-9607-4464-879c-f4bac8dff3c0\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") "
Jul 10 20:43:10.847351 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847249    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-installation-pull-secrets\") pod \"48e3380a-9607-4464-879c-f4bac8dff3c0\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") "
Jul 10 20:43:10.847351 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847281    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2595s\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-kube-api-access-2595s\") pod \"48e3380a-9607-4464-879c-f4bac8dff3c0\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") "
Jul 10 20:43:10.847351 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847321    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") pod \"48e3380a-9607-4464-879c-f4bac8dff3c0\" (UID: \"48e3380a-9607-4464-879c-f4bac8dff3c0\") "
Jul 10 20:43:10.847527 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847454    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "48e3380a-9607-4464-879c-f4bac8dff3c0" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:43:10.847657 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847624    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-trusted-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:10.848011 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.847983    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "48e3380a-9607-4464-879c-f4bac8dff3c0" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:43:10.851816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.851779    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-kube-api-access-2595s" (OuterVolumeSpecName: "kube-api-access-2595s") pod "48e3380a-9607-4464-879c-f4bac8dff3c0" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0"). InnerVolumeSpecName "kube-api-access-2595s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:43:10.851950 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.851891    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "48e3380a-9607-4464-879c-f4bac8dff3c0" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:10.852001 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.851918    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "48e3380a-9607-4464-879c-f4bac8dff3c0" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:43:10.852455 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.852396    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "48e3380a-9607-4464-879c-f4bac8dff3c0" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:10.856060 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.856000    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "48e3380a-9607-4464-879c-f4bac8dff3c0" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:43:10.860099 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.860033    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48e3380a-9607-4464-879c-f4bac8dff3c0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "48e3380a-9607-4464-879c-f4bac8dff3c0" (UID: "48e3380a-9607-4464-879c-f4bac8dff3c0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:43:10.948611 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.948545    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-image-registry-private-configuration\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:10.948611 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.948611    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e3380a-9607-4464-879c-f4bac8dff3c0-installation-pull-secrets\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:10.948816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.948628    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2595s\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-kube-api-access-2595s\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:10.948816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.948638    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-tls\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:10.948816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.948646    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e3380a-9607-4464-879c-f4bac8dff3c0-bound-sa-token\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:10.948816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.948654    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e3380a-9607-4464-879c-f4bac8dff3c0-registry-certificates\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:10.948816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:10.948662    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e3380a-9607-4464-879c-f4bac8dff3c0-ca-trust-extracted\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:11.078588 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.078527    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-p9l79" event={"ID":"bb033ea7-58e3-46e6-93b5-ed1c1b11c632","Type":"ContainerStarted","Data":"0b2e4e46ed77857ca72afa743368691dc56783326f24f3020c8df051bdf3b578"}
Jul 10 20:43:11.079328 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.079300    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-p9l79"
Jul 10 20:43:11.080271 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.080238    2565 generic.go:356] "Generic (PLEG): container finished" podID="48e3380a-9607-4464-879c-f4bac8dff3c0" containerID="3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b" exitCode=0
Jul 10 20:43:11.080397 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.080379    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt"
Jul 10 20:43:11.080896 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.080867    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt" event={"ID":"48e3380a-9607-4464-879c-f4bac8dff3c0","Type":"ContainerDied","Data":"3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b"}
Jul 10 20:43:11.081012 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.080916    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bb489dc77-mfmzt" event={"ID":"48e3380a-9607-4464-879c-f4bac8dff3c0","Type":"ContainerDied","Data":"e13d0d67b9075a03214ee5069a9097ea970b55edeed99394c2cf9e17ed43e8c6"}
Jul 10 20:43:11.081012 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.080944    2565 scope.go:117] "RemoveContainer" containerID="3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b"
Jul 10 20:43:11.083699 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.083668    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bccd78f9d-5rzpm" event={"ID":"5153ec06-6263-4270-9d0d-357307925669","Type":"ContainerStarted","Data":"369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7"}
Jul 10 20:43:11.083811 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.083722    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bccd78f9d-5rzpm" event={"ID":"5153ec06-6263-4270-9d0d-357307925669","Type":"ContainerStarted","Data":"e2d4fa21f2fa51dcc4eb333c4c30c508dad146b9e6922a6773599d81b0e53190"}
Jul 10 20:43:11.087487 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.087467    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-p9l79"
Jul 10 20:43:11.091898 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.091876    2565 scope.go:117] "RemoveContainer" containerID="3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b"
Jul 10 20:43:11.092231 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:43:11.092200    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b\": container with ID starting with 3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b not found: ID does not exist" containerID="3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b"
Jul 10 20:43:11.092302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.092238    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b"} err="failed to get container status \"3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b\": rpc error: code = NotFound desc = could not find container \"3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b\": container with ID starting with 3efa0becd569209bc5fea51a1f570940b0dd8f18e46342a35413f80db7a9658b not found: ID does not exist"
Jul 10 20:43:11.095833 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.095785    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-p9l79" podStartSLOduration=1.205801959 podStartE2EDuration="18.095767791s" totalImagesPullingTime="16.889965832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:11.095000468 +0000 UTC m=+327.587796675" watchObservedRunningTime="2026-07-10 20:43:11.095767791 +0000 UTC m=+327.588563971"
Jul 10 20:43:11.113353 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.113277    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bb489dc77-mfmzt"]
Jul 10 20:43:11.117619 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.117595    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-bb489dc77-mfmzt"]
Jul 10 20:43:11.154379 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.154317    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bccd78f9d-5rzpm" podStartSLOduration=2.15429733 podStartE2EDuration="2.15429733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:11.152943941 +0000 UTC m=+327.645740121" watchObservedRunningTime="2026-07-10 20:43:11.15429733 +0000 UTC m=+327.647093511"
Jul 10 20:43:11.958151 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:11.958115    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48e3380a-9607-4464-879c-f4bac8dff3c0" path="/var/lib/kubelet/pods/48e3380a-9607-4464-879c-f4bac8dff3c0/volumes"
Jul 10 20:43:14.106252 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:14.106156    2565 generic.go:356] "Generic (PLEG): container finished" podID="e9d30526-a2fb-40a5-8894-3dbfe93639fd" containerID="c2f57ec36c2e1975fb44614907a82ac6b0b73cf40a376c2393aa1f62a5cc24dc" exitCode=0
Jul 10 20:43:14.106696 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:14.106240    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj" event={"ID":"e9d30526-a2fb-40a5-8894-3dbfe93639fd","Type":"ContainerDied","Data":"c2f57ec36c2e1975fb44614907a82ac6b0b73cf40a376c2393aa1f62a5cc24dc"}
Jul 10 20:43:14.106696 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:14.106683    2565 scope.go:117] "RemoveContainer" containerID="c2f57ec36c2e1975fb44614907a82ac6b0b73cf40a376c2393aa1f62a5cc24dc"
Jul 10 20:43:15.112851 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:15.112812    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-22jqj" event={"ID":"e9d30526-a2fb-40a5-8894-3dbfe93639fd","Type":"ContainerStarted","Data":"ca4f450af10f47e10aad07a3227731f772a03789d4e11ffad6f5d3a6857f4d2d"}
Jul 10 20:43:20.000377 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:20.000339    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:20.000377 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:20.000398    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:20.005449 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:20.005425    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:20.139174 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:20.139148    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:43:20.185012 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:20.184980    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cddb8b78c-j5wn6"]
Jul 10 20:43:29.172777 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:29.172732    2565 generic.go:356] "Generic (PLEG): container finished" podID="73cf4177-2d24-4f8f-a408-53dac7359398" containerID="c66c17850cea952643f20dd56877f066b25ccc40a805c42e2e3a92134c314953" exitCode=0
Jul 10 20:43:29.173181 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:29.172784    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5" event={"ID":"73cf4177-2d24-4f8f-a408-53dac7359398","Type":"ContainerDied","Data":"c66c17850cea952643f20dd56877f066b25ccc40a805c42e2e3a92134c314953"}
Jul 10 20:43:29.173181 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:29.173103    2565 scope.go:117] "RemoveContainer" containerID="c66c17850cea952643f20dd56877f066b25ccc40a805c42e2e3a92134c314953"
Jul 10 20:43:30.179968 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:30.179908    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-xhsh5" event={"ID":"73cf4177-2d24-4f8f-a408-53dac7359398","Type":"ContainerStarted","Data":"eeea49705f3cfcfdb62894733f508dd1bb118992753fd348f625c6403f95976b"}
Jul 10 20:43:45.207135 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.207079    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cddb8b78c-j5wn6" podUID="484b9d14-48c3-40b7-91ab-d546b262faf5" containerName="console" containerID="cri-o://d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc" gracePeriod=15
Jul 10 20:43:45.503086 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.503057    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:43:45.554111 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.554076    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-console-config\") pod \"484b9d14-48c3-40b7-91ab-d546b262faf5\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") "
Jul 10 20:43:45.554111 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.554110    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v75dr\" (UniqueName: \"kubernetes.io/projected/484b9d14-48c3-40b7-91ab-d546b262faf5-kube-api-access-v75dr\") pod \"484b9d14-48c3-40b7-91ab-d546b262faf5\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") "
Jul 10 20:43:45.554305 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.554152    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-oauth-serving-cert\") pod \"484b9d14-48c3-40b7-91ab-d546b262faf5\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") "
Jul 10 20:43:45.554305 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.554171    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-serving-cert\") pod \"484b9d14-48c3-40b7-91ab-d546b262faf5\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") "
Jul 10 20:43:45.554305 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.554215    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-oauth-config\") pod \"484b9d14-48c3-40b7-91ab-d546b262faf5\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") "
Jul 10 20:43:45.554305 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.554257    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-service-ca\") pod \"484b9d14-48c3-40b7-91ab-d546b262faf5\" (UID: \"484b9d14-48c3-40b7-91ab-d546b262faf5\") "
Jul 10 20:43:45.554594 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.554532    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-console-config" (OuterVolumeSpecName: "console-config") pod "484b9d14-48c3-40b7-91ab-d546b262faf5" (UID: "484b9d14-48c3-40b7-91ab-d546b262faf5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:43:45.554594 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.554546    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "484b9d14-48c3-40b7-91ab-d546b262faf5" (UID: "484b9d14-48c3-40b7-91ab-d546b262faf5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:43:45.554765 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.554715    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-service-ca" (OuterVolumeSpecName: "service-ca") pod "484b9d14-48c3-40b7-91ab-d546b262faf5" (UID: "484b9d14-48c3-40b7-91ab-d546b262faf5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:43:45.556540 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.556510    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "484b9d14-48c3-40b7-91ab-d546b262faf5" (UID: "484b9d14-48c3-40b7-91ab-d546b262faf5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:45.556540 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.556513    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/484b9d14-48c3-40b7-91ab-d546b262faf5-kube-api-access-v75dr" (OuterVolumeSpecName: "kube-api-access-v75dr") pod "484b9d14-48c3-40b7-91ab-d546b262faf5" (UID: "484b9d14-48c3-40b7-91ab-d546b262faf5"). InnerVolumeSpecName "kube-api-access-v75dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:43:45.556686 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.556626    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "484b9d14-48c3-40b7-91ab-d546b262faf5" (UID: "484b9d14-48c3-40b7-91ab-d546b262faf5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:45.655390 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.655356    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-console-config\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:45.655390 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.655384    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v75dr\" (UniqueName: \"kubernetes.io/projected/484b9d14-48c3-40b7-91ab-d546b262faf5-kube-api-access-v75dr\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:45.655390 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.655393    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-oauth-serving-cert\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:45.655614 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.655402    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-serving-cert\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:45.655614 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.655412    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484b9d14-48c3-40b7-91ab-d546b262faf5-console-oauth-config\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:45.655614 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:45.655422    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484b9d14-48c3-40b7-91ab-d546b262faf5-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:46.248611 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:46.248501    2565 generic.go:356] "Generic (PLEG): container finished" podID="484b9d14-48c3-40b7-91ab-d546b262faf5" containerID="d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc" exitCode=0
Jul 10 20:43:46.248611 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:46.248552    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cddb8b78c-j5wn6"
Jul 10 20:43:46.248611 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:46.248576    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cddb8b78c-j5wn6" event={"ID":"484b9d14-48c3-40b7-91ab-d546b262faf5","Type":"ContainerDied","Data":"d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc"}
Jul 10 20:43:46.249158 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:46.248645    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cddb8b78c-j5wn6" event={"ID":"484b9d14-48c3-40b7-91ab-d546b262faf5","Type":"ContainerDied","Data":"54bf9fd673bbe10d6a5831d47ce34828004d778f08e40a64b1448646bbdca8af"}
Jul 10 20:43:46.249158 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:46.248668    2565 scope.go:117] "RemoveContainer" containerID="d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc"
Jul 10 20:43:46.256875 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:46.256861    2565 scope.go:117] "RemoveContainer" containerID="d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc"
Jul 10 20:43:46.257112 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:43:46.257095    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc\": container with ID starting with d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc not found: ID does not exist" containerID="d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc"
Jul 10 20:43:46.257181 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:46.257118    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc"} err="failed to get container status \"d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc\": rpc error: code = NotFound desc = could not find container \"d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc\": container with ID starting with d5186b2521d1b8507479f36b7d4404b90b6d1589dcd7a9079c0055f8fcccc8fc not found: ID does not exist"
Jul 10 20:43:46.267273 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:46.267245    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cddb8b78c-j5wn6"]
Jul 10 20:43:46.270878 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:46.270858    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cddb8b78c-j5wn6"]
Jul 10 20:43:47.956339 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:47.956301    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="484b9d14-48c3-40b7-91ab-d546b262faf5" path="/var/lib/kubelet/pods/484b9d14-48c3-40b7-91ab-d546b262faf5/volumes"
Jul 10 20:43:56.948102 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:56.948060    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:43:56.950716 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:56.950690    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436825a-b217-4c2d-a358-97173d8da40a-metrics-certs\") pod \"network-metrics-daemon-nbqvl\" (UID: \"5436825a-b217-4c2d-a358-97173d8da40a\") " pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:43:57.057297 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:57.057268    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n6zkh\""
Jul 10 20:43:57.062349 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:57.062334    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbqvl"
Jul 10 20:43:57.213413 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:57.213342    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nbqvl"]
Jul 10 20:43:57.216387 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:43:57.216352    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5436825a_b217_4c2d_a358_97173d8da40a.slice/crio-dafce5bc92c4a4362de5a7553b9648066d5a4f2e9baff71a2fe7f8eef0ff4aae WatchSource:0}: Error finding container dafce5bc92c4a4362de5a7553b9648066d5a4f2e9baff71a2fe7f8eef0ff4aae: Status 404 returned error can't find the container with id dafce5bc92c4a4362de5a7553b9648066d5a4f2e9baff71a2fe7f8eef0ff4aae
Jul 10 20:43:57.294347 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:57.294314    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbqvl" event={"ID":"5436825a-b217-4c2d-a358-97173d8da40a","Type":"ContainerStarted","Data":"dafce5bc92c4a4362de5a7553b9648066d5a4f2e9baff71a2fe7f8eef0ff4aae"}
Jul 10 20:43:58.217530 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:58.217498    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:43:58.217910 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:58.217823    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="alertmanager" containerID="cri-o://1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9" gracePeriod=120
Jul 10 20:43:58.217975 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:58.217898    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy-metric" containerID="cri-o://1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5" gracePeriod=120
Jul 10 20:43:58.217975 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:58.217921    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy-web" containerID="cri-o://e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45" gracePeriod=120
Jul 10 20:43:58.218070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:58.217953    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="config-reloader" containerID="cri-o://7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f" gracePeriod=120
Jul 10 20:43:58.218070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:58.218000    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="prom-label-proxy" containerID="cri-o://62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400" gracePeriod=120
Jul 10 20:43:58.218070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:58.217953    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy" containerID="cri-o://bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9" gracePeriod=120
Jul 10 20:43:59.304370 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.304339    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbqvl" event={"ID":"5436825a-b217-4c2d-a358-97173d8da40a","Type":"ContainerStarted","Data":"bd9257e97a0fc222d5a2c0012d80f7d2e429a3dac97500da0c89b338ffc6b916"}
Jul 10 20:43:59.304737 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.304404    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbqvl" event={"ID":"5436825a-b217-4c2d-a358-97173d8da40a","Type":"ContainerStarted","Data":"50baccbfaa1647f16c6b3ed132e7c52909a80649e8c8a965d77eb3ef7e2f6ff3"}
Jul 10 20:43:59.307714 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.307688    2565 generic.go:356] "Generic (PLEG): container finished" podID="44af0481-a822-4861-9199-cbb71e604a5c" containerID="62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400" exitCode=0
Jul 10 20:43:59.307714 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.307714    2565 generic.go:356] "Generic (PLEG): container finished" podID="44af0481-a822-4861-9199-cbb71e604a5c" containerID="bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9" exitCode=0
Jul 10 20:43:59.307835 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.307723    2565 generic.go:356] "Generic (PLEG): container finished" podID="44af0481-a822-4861-9199-cbb71e604a5c" containerID="7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f" exitCode=0
Jul 10 20:43:59.307835 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.307731    2565 generic.go:356] "Generic (PLEG): container finished" podID="44af0481-a822-4861-9199-cbb71e604a5c" containerID="1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9" exitCode=0
Jul 10 20:43:59.307835 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.307745    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerDied","Data":"62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400"}
Jul 10 20:43:59.307835 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.307802    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerDied","Data":"bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9"}
Jul 10 20:43:59.307835 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.307827    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerDied","Data":"7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f"}
Jul 10 20:43:59.307980 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.307850    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerDied","Data":"1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9"}
Jul 10 20:43:59.321446 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.321405    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nbqvl" podStartSLOduration=373.90126061 podStartE2EDuration="6m15.321392648s" totalImagesPullingTime="1.420132038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:59.31926901 +0000 UTC m=+375.812065209" watchObservedRunningTime="2026-07-10 20:43:59.321392648 +0000 UTC m=+375.814188827"
Jul 10 20:43:59.473231 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.473202    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:43:59.571646 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571620    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-main-db\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.571771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571658    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.571771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571696    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-config-out\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.571771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571721    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-tls-assets\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.571771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571740    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nl895\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-kube-api-access-nl895\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.571771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571766    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-web-config\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.571995 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571806    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-metrics-client-ca\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.571995 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571905    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.571995 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571952    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-config-volume\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.571995 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571977    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-cluster-tls-config\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.572181 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.571984    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:43:59.572181 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.572009    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-trusted-ca-bundle\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.572181 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.572052    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-main-tls\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.572181 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.572097    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-web\") pod \"44af0481-a822-4861-9199-cbb71e604a5c\" (UID: \"44af0481-a822-4861-9199-cbb71e604a5c\") "
Jul 10 20:43:59.572845 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.572474    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-main-db\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.572975 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.572777    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:43:59.573743 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.573711    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:43:59.574645 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.574623    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-config-volume" (OuterVolumeSpecName: "config-volume") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:59.575447 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.575345    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-config-out" (OuterVolumeSpecName: "config-out") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:43:59.575547 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.575513    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:43:59.575636 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.575580    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:59.575978 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.575957    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:59.576651 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.576625    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:59.576735 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.576646    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-kube-api-access-nl895" (OuterVolumeSpecName: "kube-api-access-nl895") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "kube-api-access-nl895". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:43:59.577396 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.577376    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:59.579699 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.579677    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:59.586926 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.586905    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-web-config" (OuterVolumeSpecName: "web-config") pod "44af0481-a822-4861-9199-cbb71e604a5c" (UID: "44af0481-a822-4861-9199-cbb71e604a5c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:43:59.673415 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673392    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl895\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-kube-api-access-nl895\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673415 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673413    2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-web-config\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673422    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-metrics-client-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673432    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673440    2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-config-volume\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673448    2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-cluster-tls-config\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673456    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44af0481-a822-4861-9199-cbb71e604a5c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673465    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-main-tls\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673473    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673482    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/44af0481-a822-4861-9199-cbb71e604a5c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673490    2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44af0481-a822-4861-9199-cbb71e604a5c-config-out\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:43:59.673536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:43:59.673498    2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44af0481-a822-4861-9199-cbb71e604a5c-tls-assets\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:44:00.314848 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.314814    2565 generic.go:356] "Generic (PLEG): container finished" podID="44af0481-a822-4861-9199-cbb71e604a5c" containerID="1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5" exitCode=0
Jul 10 20:44:00.314848 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.314840    2565 generic.go:356] "Generic (PLEG): container finished" podID="44af0481-a822-4861-9199-cbb71e604a5c" containerID="e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45" exitCode=0
Jul 10 20:44:00.315292 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.314902    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.315292 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.314900    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerDied","Data":"1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5"}
Jul 10 20:44:00.315292 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.314989    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerDied","Data":"e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45"}
Jul 10 20:44:00.315292 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.315030    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"44af0481-a822-4861-9199-cbb71e604a5c","Type":"ContainerDied","Data":"8f08889c76b3c20ebdaf5997e6ecf370f277e8af0af80f2494c675218d49abfa"}
Jul 10 20:44:00.315292 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.315055    2565 scope.go:117] "RemoveContainer" containerID="62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400"
Jul 10 20:44:00.323189 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.323164    2565 scope.go:117] "RemoveContainer" containerID="1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5"
Jul 10 20:44:00.330223 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.330209    2565 scope.go:117] "RemoveContainer" containerID="bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9"
Jul 10 20:44:00.338016 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.337878    2565 scope.go:117] "RemoveContainer" containerID="e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45"
Jul 10 20:44:00.338016 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.337924    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:44:00.342018 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.341997    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:44:00.344816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.344780    2565 scope.go:117] "RemoveContainer" containerID="7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f"
Jul 10 20:44:00.351735 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.351720    2565 scope.go:117] "RemoveContainer" containerID="1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9"
Jul 10 20:44:00.358950 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.358935    2565 scope.go:117] "RemoveContainer" containerID="e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b"
Jul 10 20:44:00.366504 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.366487    2565 scope.go:117] "RemoveContainer" containerID="62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400"
Jul 10 20:44:00.366848 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:44:00.366832    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400\": container with ID starting with 62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400 not found: ID does not exist" containerID="62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400"
Jul 10 20:44:00.366924 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.366856    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400"} err="failed to get container status \"62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400\": rpc error: code = NotFound desc = could not find container \"62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400\": container with ID starting with 62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400 not found: ID does not exist"
Jul 10 20:44:00.366924 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.366875    2565 scope.go:117] "RemoveContainer" containerID="1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5"
Jul 10 20:44:00.367177 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:44:00.367152    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5\": container with ID starting with 1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5 not found: ID does not exist" containerID="1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5"
Jul 10 20:44:00.367224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367188    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5"} err="failed to get container status \"1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5\": rpc error: code = NotFound desc = could not find container \"1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5\": container with ID starting with 1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5 not found: ID does not exist"
Jul 10 20:44:00.367224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367209    2565 scope.go:117] "RemoveContainer" containerID="bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9"
Jul 10 20:44:00.367306 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367259    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:44:00.367500 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:44:00.367474    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9\": container with ID starting with bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9 not found: ID does not exist" containerID="bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9"
Jul 10 20:44:00.367595 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367509    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9"} err="failed to get container status \"bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9\": rpc error: code = NotFound desc = could not find container \"bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9\": container with ID starting with bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9 not found: ID does not exist"
Jul 10 20:44:00.367595 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367531    2565 scope.go:117] "RemoveContainer" containerID="e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367594    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="prom-label-proxy"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367607    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="prom-label-proxy"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367617    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="init-config-reloader"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367623    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="init-config-reloader"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367632    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="alertmanager"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367641    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="alertmanager"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367657    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy-web"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367665    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy-web"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367674    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="484b9d14-48c3-40b7-91ab-d546b262faf5" containerName="console"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367682    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="484b9d14-48c3-40b7-91ab-d546b262faf5" containerName="console"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367703    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy-metric"
Jul 10 20:44:00.367704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367711    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy-metric"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367721    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e3380a-9607-4464-879c-f4bac8dff3c0" containerName="registry"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367727    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e3380a-9607-4464-879c-f4bac8dff3c0" containerName="registry"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367738    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="config-reloader"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367744    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="config-reloader"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367750    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367755    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367830    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy-metric"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367842    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:44:00.367839    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45\": container with ID starting with e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45 not found: ID does not exist" containerID="e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367851    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="config-reloader"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367860    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="kube-rbac-proxy-web"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367866    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="alertmanager"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367871    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="484b9d14-48c3-40b7-91ab-d546b262faf5" containerName="console"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367875    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="48e3380a-9607-4464-879c-f4bac8dff3c0" containerName="registry"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367881    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="44af0481-a822-4861-9199-cbb71e604a5c" containerName="prom-label-proxy"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367866    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45"} err="failed to get container status \"e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45\": rpc error: code = NotFound desc = could not find container \"e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45\": container with ID starting with e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45 not found: ID does not exist"
Jul 10 20:44:00.368224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.367954    2565 scope.go:117] "RemoveContainer" containerID="7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f"
Jul 10 20:44:00.369058 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:44:00.368252    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f\": container with ID starting with 7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f not found: ID does not exist" containerID="7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f"
Jul 10 20:44:00.369058 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.368281    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f"} err="failed to get container status \"7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f\": rpc error: code = NotFound desc = could not find container \"7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f\": container with ID starting with 7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f not found: ID does not exist"
Jul 10 20:44:00.369058 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.368304    2565 scope.go:117] "RemoveContainer" containerID="1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9"
Jul 10 20:44:00.369058 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:44:00.368548    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9\": container with ID starting with 1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9 not found: ID does not exist" containerID="1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9"
Jul 10 20:44:00.369058 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.368596    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9"} err="failed to get container status \"1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9\": rpc error: code = NotFound desc = could not find container \"1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9\": container with ID starting with 1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9 not found: ID does not exist"
Jul 10 20:44:00.369058 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.368615    2565 scope.go:117] "RemoveContainer" containerID="e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b"
Jul 10 20:44:00.369312 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:44:00.369063    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b\": container with ID starting with e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b not found: ID does not exist" containerID="e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b"
Jul 10 20:44:00.369312 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.369091    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b"} err="failed to get container status \"e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b\": rpc error: code = NotFound desc = could not find container \"e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b\": container with ID starting with e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b not found: ID does not exist"
Jul 10 20:44:00.369312 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.369112    2565 scope.go:117] "RemoveContainer" containerID="62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400"
Jul 10 20:44:00.369739 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.369708    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400"} err="failed to get container status \"62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400\": rpc error: code = NotFound desc = could not find container \"62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400\": container with ID starting with 62b3a983e00df9cac5435ee46cef6b8f6af5494ad35f757709100088becd7400 not found: ID does not exist"
Jul 10 20:44:00.369739 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.369737    2565 scope.go:117] "RemoveContainer" containerID="1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5"
Jul 10 20:44:00.370004 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.369980    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5"} err="failed to get container status \"1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5\": rpc error: code = NotFound desc = could not find container \"1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5\": container with ID starting with 1aeb88f8e4da95224aed6a122c3fe640030a751a2d84d992f2ca8795b2ae6fc5 not found: ID does not exist"
Jul 10 20:44:00.370004 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.370004    2565 scope.go:117] "RemoveContainer" containerID="bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9"
Jul 10 20:44:00.370283 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.370255    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9"} err="failed to get container status \"bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9\": rpc error: code = NotFound desc = could not find container \"bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9\": container with ID starting with bccc84043fdc582e5ff0a25d6cb61dd59b9ad6a8f0d3dd93a47752b4335427b9 not found: ID does not exist"
Jul 10 20:44:00.370366 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.370283    2565 scope.go:117] "RemoveContainer" containerID="e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45"
Jul 10 20:44:00.371577 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.371525    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45"} err="failed to get container status \"e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45\": rpc error: code = NotFound desc = could not find container \"e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45\": container with ID starting with e4129d4200583a0deda46798109c0fca24102e16f09ad4a797d6b3c4b49a2d45 not found: ID does not exist"
Jul 10 20:44:00.371673 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.371589    2565 scope.go:117] "RemoveContainer" containerID="7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f"
Jul 10 20:44:00.371855 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.371831    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f"} err="failed to get container status \"7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f\": rpc error: code = NotFound desc = could not find container \"7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f\": container with ID starting with 7bc0842b7aa9fde5e15fc503dff4ad1da6956a52b02462d5cfc3abc35640b78f not found: ID does not exist"
Jul 10 20:44:00.371907 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.371859    2565 scope.go:117] "RemoveContainer" containerID="1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9"
Jul 10 20:44:00.372160 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.372137    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9"} err="failed to get container status \"1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9\": rpc error: code = NotFound desc = could not find container \"1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9\": container with ID starting with 1bf19a265d68dc3b28110d168a5e74d81fc58da6f90d65f8f2751131f2a725a9 not found: ID does not exist"
Jul 10 20:44:00.372231 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.372162    2565 scope.go:117] "RemoveContainer" containerID="e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b"
Jul 10 20:44:00.372433 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.372409    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b"} err="failed to get container status \"e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b\": rpc error: code = NotFound desc = could not find container \"e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b\": container with ID starting with e8d655d832556a95a3253538414598ec5cdf2736f906f9f13891fbff68e0c52b not found: ID does not exist"
Jul 10 20:44:00.372872 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.372844    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.375466 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.375450    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 20:44:00.375621 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.375466    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 20:44:00.375621 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.375608    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 20:44:00.375741 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.375642    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 20:44:00.375741 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.375650    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 20:44:00.376072 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.376057    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 20:44:00.376156 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.376054    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 20:44:00.376156 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.376094    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7chfv\""
Jul 10 20:44:00.376156 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.376135    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 20:44:00.381177 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.381158    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 20:44:00.386380 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.386349    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:44:00.480038 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.479996    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480038 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480039    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crf28\" (UniqueName: \"kubernetes.io/projected/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-kube-api-access-crf28\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480234 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480074    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480234 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480234 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480166    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480234 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480191    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480234 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480217    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480387 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480264    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-web-config\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480387 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480292    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-config-out\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480387 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480387 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480333    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480387 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480355    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.480387 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.480375    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581117 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581081    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581121    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-web-config\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581146    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-config-out\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581169    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581207    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581240    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581278    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581311    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581333    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crf28\" (UniqueName: \"kubernetes.io/projected/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-kube-api-access-crf28\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.581302 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581360    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.583652 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581382    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.583652 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581404    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.583652 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581433    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.583652 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.581703    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.583652 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.582745    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.584536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.584353    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.584536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.584472    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-web-config\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.584536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.584472    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.584795 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.584548    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.584795 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.584615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.584941 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.584917    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-config-out\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.585012 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.584949    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.585149 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.585129    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.585789 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.585767    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.586739 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.586722    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.591922 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.591905    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crf28\" (UniqueName: \"kubernetes.io/projected/e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4-kube-api-access-crf28\") pod \"alertmanager-main-0\" (UID: \"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.684705 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.684675    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:44:00.823347 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:00.823315    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:44:00.826296 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:44:00.826254    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80e7205_76fb_4c1a_9dcf_c387f7dfb1a4.slice/crio-225f49a862d463c6e1acf58cd99d83121e9229ca3a7add8cc4a72c9400091b56 WatchSource:0}: Error finding container 225f49a862d463c6e1acf58cd99d83121e9229ca3a7add8cc4a72c9400091b56: Status 404 returned error can't find the container with id 225f49a862d463c6e1acf58cd99d83121e9229ca3a7add8cc4a72c9400091b56
Jul 10 20:44:01.322055 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.322016    2565 generic.go:356] "Generic (PLEG): container finished" podID="e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4" containerID="4594eb7f8660a34926950741bd882255fe8449c5ab932febbab89aee3153e682" exitCode=0
Jul 10 20:44:01.322423 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.322110    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4","Type":"ContainerDied","Data":"4594eb7f8660a34926950741bd882255fe8449c5ab932febbab89aee3153e682"}
Jul 10 20:44:01.322423 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.322173    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4","Type":"ContainerStarted","Data":"225f49a862d463c6e1acf58cd99d83121e9229ca3a7add8cc4a72c9400091b56"}
Jul 10 20:44:01.717605 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.717546    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bf64dbcc-nftsc"]
Jul 10 20:44:01.721142 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.721117    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.733771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.733745    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bf64dbcc-nftsc"]
Jul 10 20:44:01.791807 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.791776    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-oauth-config\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.791807 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.791806    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-service-ca\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.791968 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.791834    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-oauth-serving-cert\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.791968 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.791924    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-serving-cert\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.791968 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.791956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-console-config\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.792076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.792018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-trusted-ca-bundle\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.792076 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.792037    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxk7q\" (UniqueName: \"kubernetes.io/projected/11590b47-f7fa-45a2-aa30-10dee877e842-kube-api-access-wxk7q\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.892787 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.892682    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-oauth-serving-cert\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.892943 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.892779    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-serving-cert\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.892943 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.892830    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-console-config\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.892943 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.892874    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-trusted-ca-bundle\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.892943 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.892898    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxk7q\" (UniqueName: \"kubernetes.io/projected/11590b47-f7fa-45a2-aa30-10dee877e842-kube-api-access-wxk7q\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.893124 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.892958    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-oauth-config\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.893124 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.892984    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-service-ca\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.893635 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.893607    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-console-config\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.893635 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.893639    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-oauth-serving-cert\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.893916 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.893700    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-service-ca\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.893916 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.893864    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-trusted-ca-bundle\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.895462 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.895439    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-serving-cert\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.895552 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.895497    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-oauth-config\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.900809 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.900788    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxk7q\" (UniqueName: \"kubernetes.io/projected/11590b47-f7fa-45a2-aa30-10dee877e842-kube-api-access-wxk7q\") pod \"console-bf64dbcc-nftsc\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") " pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:01.957200 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:01.957159    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44af0481-a822-4861-9199-cbb71e604a5c" path="/var/lib/kubelet/pods/44af0481-a822-4861-9199-cbb71e604a5c/volumes"
Jul 10 20:44:02.030496 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.030463    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:02.161492 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.161469    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bf64dbcc-nftsc"]
Jul 10 20:44:02.163947 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:44:02.163916    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11590b47_f7fa_45a2_aa30_10dee877e842.slice/crio-7670d3143f8c68f96052b1557fa038d5e1c86c069ac21a6616d177680a83edaf WatchSource:0}: Error finding container 7670d3143f8c68f96052b1557fa038d5e1c86c069ac21a6616d177680a83edaf: Status 404 returned error can't find the container with id 7670d3143f8c68f96052b1557fa038d5e1c86c069ac21a6616d177680a83edaf
Jul 10 20:44:02.331061 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.330870    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4","Type":"ContainerStarted","Data":"6db772fbcfb5db06e99b3b2bf413494669e2fd5128c1b745e171d9d26b26642f"}
Jul 10 20:44:02.331061 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.330955    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4","Type":"ContainerStarted","Data":"88a7608b0c368ea823408069cf4ad5aac90117008423cb98ee02e724d98bcb15"}
Jul 10 20:44:02.331061 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.330992    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4","Type":"ContainerStarted","Data":"598baaeed0b6e5afa6c16273030f7cac00a6341984002e25948704628ff06a0a"}
Jul 10 20:44:02.331061 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.331031    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4","Type":"ContainerStarted","Data":"0c5d5bb85ad22fc10aabc1efabd83582f782dc5f4c9bb807c2252c5e1b603133"}
Jul 10 20:44:02.331061 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.331066    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4","Type":"ContainerStarted","Data":"b0866896ebb755555cd95a57c5c08a08971d99aa97cf79445e9009e329e48b34"}
Jul 10 20:44:02.331710 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.331092    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4","Type":"ContainerStarted","Data":"26e2adfb6b085056f46bae70f61e273ff2aa6ddee639dc42075d290e2b8b0389"}
Jul 10 20:44:02.333369 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.333344    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf64dbcc-nftsc" event={"ID":"11590b47-f7fa-45a2-aa30-10dee877e842","Type":"ContainerStarted","Data":"5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793"}
Jul 10 20:44:02.333484 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.333381    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf64dbcc-nftsc" event={"ID":"11590b47-f7fa-45a2-aa30-10dee877e842","Type":"ContainerStarted","Data":"7670d3143f8c68f96052b1557fa038d5e1c86c069ac21a6616d177680a83edaf"}
Jul 10 20:44:02.359871 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.359805    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.344213002 podStartE2EDuration="2.359782433s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.569431ms" isStatefulPod=false podCreationTimestamp="2026-07-10 20:44:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:44:02.357010493 +0000 UTC m=+378.849806673" watchObservedRunningTime="2026-07-10 20:44:02.359782433 +0000 UTC m=+378.852578613"
Jul 10 20:44:02.375908 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:02.375827    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bf64dbcc-nftsc" podStartSLOduration=1.3757842120000001 podStartE2EDuration="1.375784212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:44:02.372950569 +0000 UTC m=+378.865746749" watchObservedRunningTime="2026-07-10 20:44:02.375784212 +0000 UTC m=+378.868580393"
Jul 10 20:44:12.031379 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:12.031347    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:12.031795 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:12.031400    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:12.035971 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:12.035949    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:12.382368 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:12.382339    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:44:12.425825 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:12.425798    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bccd78f9d-5rzpm"]
Jul 10 20:44:37.446996 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.446950    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bccd78f9d-5rzpm" podUID="5153ec06-6263-4270-9d0d-357307925669" containerName="console" containerID="cri-o://369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7" gracePeriod=15
Jul 10 20:44:37.689401 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.689373    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:44:37.790379 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790309    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-trusted-ca-bundle\") pod \"5153ec06-6263-4270-9d0d-357307925669\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") "
Jul 10 20:44:37.790379 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790346    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-serving-cert\") pod \"5153ec06-6263-4270-9d0d-357307925669\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") "
Jul 10 20:44:37.790619 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790402    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-oauth-serving-cert\") pod \"5153ec06-6263-4270-9d0d-357307925669\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") "
Jul 10 20:44:37.790619 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790422    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-service-ca\") pod \"5153ec06-6263-4270-9d0d-357307925669\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") "
Jul 10 20:44:37.790619 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790440    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tjx8\" (UniqueName: \"kubernetes.io/projected/5153ec06-6263-4270-9d0d-357307925669-kube-api-access-4tjx8\") pod \"5153ec06-6263-4270-9d0d-357307925669\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") "
Jul 10 20:44:37.790619 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790498    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-oauth-config\") pod \"5153ec06-6263-4270-9d0d-357307925669\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") "
Jul 10 20:44:37.790619 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790522    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-console-config\") pod \"5153ec06-6263-4270-9d0d-357307925669\" (UID: \"5153ec06-6263-4270-9d0d-357307925669\") "
Jul 10 20:44:37.790872 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790842    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5153ec06-6263-4270-9d0d-357307925669" (UID: "5153ec06-6263-4270-9d0d-357307925669"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:44:37.790969 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790883    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5153ec06-6263-4270-9d0d-357307925669" (UID: "5153ec06-6263-4270-9d0d-357307925669"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:44:37.790969 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.790959    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-service-ca" (OuterVolumeSpecName: "service-ca") pod "5153ec06-6263-4270-9d0d-357307925669" (UID: "5153ec06-6263-4270-9d0d-357307925669"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:44:37.791107 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.791077    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-console-config" (OuterVolumeSpecName: "console-config") pod "5153ec06-6263-4270-9d0d-357307925669" (UID: "5153ec06-6263-4270-9d0d-357307925669"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:44:37.792762 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.792725    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5153ec06-6263-4270-9d0d-357307925669" (UID: "5153ec06-6263-4270-9d0d-357307925669"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:44:37.792762 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.792739    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5153ec06-6263-4270-9d0d-357307925669" (UID: "5153ec06-6263-4270-9d0d-357307925669"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:44:37.792908 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.792847    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5153ec06-6263-4270-9d0d-357307925669-kube-api-access-4tjx8" (OuterVolumeSpecName: "kube-api-access-4tjx8") pod "5153ec06-6263-4270-9d0d-357307925669" (UID: "5153ec06-6263-4270-9d0d-357307925669"). InnerVolumeSpecName "kube-api-access-4tjx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:44:37.891274 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.891243    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-oauth-config\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:44:37.891274 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.891269    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-console-config\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:44:37.891274 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.891279    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-trusted-ca-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:44:37.891461 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.891288    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5153ec06-6263-4270-9d0d-357307925669-console-serving-cert\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:44:37.891461 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.891297    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-oauth-serving-cert\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:44:37.891461 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.891304    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5153ec06-6263-4270-9d0d-357307925669-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:44:37.891461 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:37.891312    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tjx8\" (UniqueName: \"kubernetes.io/projected/5153ec06-6263-4270-9d0d-357307925669-kube-api-access-4tjx8\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:44:38.484496 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:38.484463    2565 generic.go:356] "Generic (PLEG): container finished" podID="5153ec06-6263-4270-9d0d-357307925669" containerID="369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7" exitCode=0
Jul 10 20:44:38.484951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:38.484509    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bccd78f9d-5rzpm"
Jul 10 20:44:38.484951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:38.484546    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bccd78f9d-5rzpm" event={"ID":"5153ec06-6263-4270-9d0d-357307925669","Type":"ContainerDied","Data":"369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7"}
Jul 10 20:44:38.484951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:38.484614    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bccd78f9d-5rzpm" event={"ID":"5153ec06-6263-4270-9d0d-357307925669","Type":"ContainerDied","Data":"e2d4fa21f2fa51dcc4eb333c4c30c508dad146b9e6922a6773599d81b0e53190"}
Jul 10 20:44:38.484951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:38.484635    2565 scope.go:117] "RemoveContainer" containerID="369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7"
Jul 10 20:44:38.492941 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:38.492923    2565 scope.go:117] "RemoveContainer" containerID="369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7"
Jul 10 20:44:38.493209 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:44:38.493188    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7\": container with ID starting with 369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7 not found: ID does not exist" containerID="369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7"
Jul 10 20:44:38.493305 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:38.493215    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7"} err="failed to get container status \"369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7\": rpc error: code = NotFound desc = could not find container \"369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7\": container with ID starting with 369a3e8b9a39db68573290cdc986c789365e24b1dd431548480716ec70ba28f7 not found: ID does not exist"
Jul 10 20:44:38.500981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:38.500955    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bccd78f9d-5rzpm"]
Jul 10 20:44:38.504737 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:38.504717    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bccd78f9d-5rzpm"]
Jul 10 20:44:39.955751 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:44:39.955718    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5153ec06-6263-4270-9d0d-357307925669" path="/var/lib/kubelet/pods/5153ec06-6263-4270-9d0d-357307925669/volumes"
Jul 10 20:45:04.518981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.518868    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"]
Jul 10 20:45:04.519448 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.519343    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5153ec06-6263-4270-9d0d-357307925669" containerName="console"
Jul 10 20:45:04.519448 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.519362    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5153ec06-6263-4270-9d0d-357307925669" containerName="console"
Jul 10 20:45:04.519593 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.519462    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5153ec06-6263-4270-9d0d-357307925669" containerName="console"
Jul 10 20:45:04.523965 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.523944    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.526691 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.526672    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 10 20:45:04.527003 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.526980    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 10 20:45:04.528172 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.528116    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-td5xw\""
Jul 10 20:45:04.530153 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.530104    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"]
Jul 10 20:45:04.599578 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.599523    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvkgm\" (UniqueName: \"kubernetes.io/projected/0afe1c30-a2af-442b-807b-d49ec53787b7-kube-api-access-dvkgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.599738 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.599632    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.599738 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.599671    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.700840 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.700787    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.701021 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.700854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.701021 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.700920    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvkgm\" (UniqueName: \"kubernetes.io/projected/0afe1c30-a2af-442b-807b-d49ec53787b7-kube-api-access-dvkgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.701184 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.701162    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.701248 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.701191    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.709399 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.709373    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvkgm\" (UniqueName: \"kubernetes.io/projected/0afe1c30-a2af-442b-807b-d49ec53787b7-kube-api-access-dvkgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.834388 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.834346    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:04.974840 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:04.974804    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"]
Jul 10 20:45:04.982436 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:45:04.982411    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0afe1c30_a2af_442b_807b_d49ec53787b7.slice/crio-64f23d7c788d0ed0cf55882c698964a71534e36dc0110dc8369c997e5d2a22fd WatchSource:0}: Error finding container 64f23d7c788d0ed0cf55882c698964a71534e36dc0110dc8369c997e5d2a22fd: Status 404 returned error can't find the container with id 64f23d7c788d0ed0cf55882c698964a71534e36dc0110dc8369c997e5d2a22fd
Jul 10 20:45:05.591451 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:05.591412    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8" event={"ID":"0afe1c30-a2af-442b-807b-d49ec53787b7","Type":"ContainerStarted","Data":"64f23d7c788d0ed0cf55882c698964a71534e36dc0110dc8369c997e5d2a22fd"}
Jul 10 20:45:11.620948 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:11.620909    2565 generic.go:356] "Generic (PLEG): container finished" podID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerID="a754d9cc76c95f475f7c209d689a4720003b66f627d4348eb4e418e3be695ea0" exitCode=0
Jul 10 20:45:11.621404 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:11.620957    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8" event={"ID":"0afe1c30-a2af-442b-807b-d49ec53787b7","Type":"ContainerDied","Data":"a754d9cc76c95f475f7c209d689a4720003b66f627d4348eb4e418e3be695ea0"}
Jul 10 20:45:14.639138 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:14.639103    2565 generic.go:356] "Generic (PLEG): container finished" podID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerID="8cdadfebff0eff6fb7756657e585931ab613f3352e5358d0f0be1ac347c7529d" exitCode=0
Jul 10 20:45:14.639655 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:14.639151    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8" event={"ID":"0afe1c30-a2af-442b-807b-d49ec53787b7","Type":"ContainerDied","Data":"8cdadfebff0eff6fb7756657e585931ab613f3352e5358d0f0be1ac347c7529d"}
Jul 10 20:45:22.677029 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:22.676990    2565 generic.go:356] "Generic (PLEG): container finished" podID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerID="3fb0fc96a19f0a08020050bee0ab24a6b9246c01a670eb4968edec824985a11c" exitCode=0
Jul 10 20:45:22.677413 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:22.677047    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8" event={"ID":"0afe1c30-a2af-442b-807b-d49ec53787b7","Type":"ContainerDied","Data":"3fb0fc96a19f0a08020050bee0ab24a6b9246c01a670eb4968edec824985a11c"}
Jul 10 20:45:23.807213 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.807185    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:23.874307 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.874269    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-bundle\") pod \"0afe1c30-a2af-442b-807b-d49ec53787b7\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") "
Jul 10 20:45:23.874307 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.874316    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvkgm\" (UniqueName: \"kubernetes.io/projected/0afe1c30-a2af-442b-807b-d49ec53787b7-kube-api-access-dvkgm\") pod \"0afe1c30-a2af-442b-807b-d49ec53787b7\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") "
Jul 10 20:45:23.874502 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.874344    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-util\") pod \"0afe1c30-a2af-442b-807b-d49ec53787b7\" (UID: \"0afe1c30-a2af-442b-807b-d49ec53787b7\") "
Jul 10 20:45:23.875039 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.875007    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-bundle" (OuterVolumeSpecName: "bundle") pod "0afe1c30-a2af-442b-807b-d49ec53787b7" (UID: "0afe1c30-a2af-442b-807b-d49ec53787b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:45:23.876731 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.876705    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0afe1c30-a2af-442b-807b-d49ec53787b7-kube-api-access-dvkgm" (OuterVolumeSpecName: "kube-api-access-dvkgm") pod "0afe1c30-a2af-442b-807b-d49ec53787b7" (UID: "0afe1c30-a2af-442b-807b-d49ec53787b7"). InnerVolumeSpecName "kube-api-access-dvkgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:45:23.880224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.880202    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-util" (OuterVolumeSpecName: "util") pod "0afe1c30-a2af-442b-807b-d49ec53787b7" (UID: "0afe1c30-a2af-442b-807b-d49ec53787b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:45:23.975173 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.975147    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvkgm\" (UniqueName: \"kubernetes.io/projected/0afe1c30-a2af-442b-807b-d49ec53787b7-kube-api-access-dvkgm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:45:23.975173 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.975171    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-util\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:45:23.975318 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:23.975179    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0afe1c30-a2af-442b-807b-d49ec53787b7-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:45:24.686581 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:24.686537    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8"
Jul 10 20:45:24.686759 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:24.686532    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409ngw8" event={"ID":"0afe1c30-a2af-442b-807b-d49ec53787b7","Type":"ContainerDied","Data":"64f23d7c788d0ed0cf55882c698964a71534e36dc0110dc8369c997e5d2a22fd"}
Jul 10 20:45:24.686759 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:24.686692    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64f23d7c788d0ed0cf55882c698964a71534e36dc0110dc8369c997e5d2a22fd"
Jul 10 20:45:32.272370 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.272334    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"]
Jul 10 20:45:32.272981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.272812    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerName="extract"
Jul 10 20:45:32.272981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.272833    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerName="extract"
Jul 10 20:45:32.272981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.272874    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerName="util"
Jul 10 20:45:32.272981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.272882    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerName="util"
Jul 10 20:45:32.272981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.272893    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerName="pull"
Jul 10 20:45:32.272981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.272901    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerName="pull"
Jul 10 20:45:32.272981 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.272980    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0afe1c30-a2af-442b-807b-d49ec53787b7" containerName="extract"
Jul 10 20:45:32.277936 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.277915    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"
Jul 10 20:45:32.280883 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.280859    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:45:32.281005 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.280901    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 10 20:45:32.281078 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.281035    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-kcbjr\""
Jul 10 20:45:32.288413 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.288390    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"]
Jul 10 20:45:32.339517 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.339485    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5tcm\" (UniqueName: \"kubernetes.io/projected/a106765e-6713-48aa-b563-1fd2555dac9d-kube-api-access-d5tcm\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxf2h\" (UID: \"a106765e-6713-48aa-b563-1fd2555dac9d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"
Jul 10 20:45:32.339691 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.339535    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a106765e-6713-48aa-b563-1fd2555dac9d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxf2h\" (UID: \"a106765e-6713-48aa-b563-1fd2555dac9d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"
Jul 10 20:45:32.440700 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.440666    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5tcm\" (UniqueName: \"kubernetes.io/projected/a106765e-6713-48aa-b563-1fd2555dac9d-kube-api-access-d5tcm\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxf2h\" (UID: \"a106765e-6713-48aa-b563-1fd2555dac9d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"
Jul 10 20:45:32.440875 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.440721    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a106765e-6713-48aa-b563-1fd2555dac9d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxf2h\" (UID: \"a106765e-6713-48aa-b563-1fd2555dac9d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"
Jul 10 20:45:32.441135 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.441119    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a106765e-6713-48aa-b563-1fd2555dac9d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxf2h\" (UID: \"a106765e-6713-48aa-b563-1fd2555dac9d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"
Jul 10 20:45:32.456666 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.456637    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5tcm\" (UniqueName: \"kubernetes.io/projected/a106765e-6713-48aa-b563-1fd2555dac9d-kube-api-access-d5tcm\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxf2h\" (UID: \"a106765e-6713-48aa-b563-1fd2555dac9d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"
Jul 10 20:45:32.587289 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.587246    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"
Jul 10 20:45:32.727734 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:32.727707    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h"]
Jul 10 20:45:32.730577 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:45:32.730532    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda106765e_6713_48aa_b563_1fd2555dac9d.slice/crio-78443f732776ca3d0216ce718f14a2b15b7ce934661ea6966f35c3e17e5f5cca WatchSource:0}: Error finding container 78443f732776ca3d0216ce718f14a2b15b7ce934661ea6966f35c3e17e5f5cca: Status 404 returned error can't find the container with id 78443f732776ca3d0216ce718f14a2b15b7ce934661ea6966f35c3e17e5f5cca
Jul 10 20:45:33.726199 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:33.726155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h" event={"ID":"a106765e-6713-48aa-b563-1fd2555dac9d","Type":"ContainerStarted","Data":"78443f732776ca3d0216ce718f14a2b15b7ce934661ea6966f35c3e17e5f5cca"}
Jul 10 20:45:34.732667 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:34.732546    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h" event={"ID":"a106765e-6713-48aa-b563-1fd2555dac9d","Type":"ContainerStarted","Data":"9bcf7da0991bb9d308e57217b0536576bbb7fcda1ac550d8b39bd0efea82e95d"}
Jul 10 20:45:34.755528 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:34.754950    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxf2h" podStartSLOduration=1.100945528 podStartE2EDuration="2.75493237s" totalImagesPullingTime="1.653986842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:34.754035767 +0000 UTC m=+471.246831946" watchObservedRunningTime="2026-07-10 20:45:34.75493237 +0000 UTC m=+471.247728550"
Jul 10 20:45:35.961923 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:35.961893    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"]
Jul 10 20:45:35.965381 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:35.965359    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:35.968318 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:35.968298    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 10 20:45:35.969549 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:35.969533    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-td5xw\""
Jul 10 20:45:35.969625 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:35.969571    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 10 20:45:35.973540 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:35.973519    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"]
Jul 10 20:45:36.068464 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.068431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7tbz\" (UniqueName: \"kubernetes.io/projected/a7460680-f9f8-4821-a9bc-6e2bbc52c670-kube-api-access-k7tbz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.068464 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.068465    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.068677 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.068576    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.169878 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.169843    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbz\" (UniqueName: \"kubernetes.io/projected/a7460680-f9f8-4821-a9bc-6e2bbc52c670-kube-api-access-k7tbz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.169878 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.169880    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.170054 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.169958    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.170339 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.170317    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.170401 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.170335    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.177914 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.177891    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7tbz\" (UniqueName: \"kubernetes.io/projected/a7460680-f9f8-4821-a9bc-6e2bbc52c670-kube-api-access-k7tbz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.274685 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.274601    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:36.408190 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.408154    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"]
Jul 10 20:45:36.411182 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:45:36.411156    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7460680_f9f8_4821_a9bc_6e2bbc52c670.slice/crio-446f276c625fb5131b970dd86b8bb26928861b2d8739d1a0fcb9008956074eae WatchSource:0}: Error finding container 446f276c625fb5131b970dd86b8bb26928861b2d8739d1a0fcb9008956074eae: Status 404 returned error can't find the container with id 446f276c625fb5131b970dd86b8bb26928861b2d8739d1a0fcb9008956074eae
Jul 10 20:45:36.742210 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.742175    2565 generic.go:356] "Generic (PLEG): container finished" podID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerID="b1443f6a1ce8f6365763c31d8bcb62fd31fe1a0783f447264ee4483b1aaeb6fa" exitCode=0
Jul 10 20:45:36.742210 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.742211    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8" event={"ID":"a7460680-f9f8-4821-a9bc-6e2bbc52c670","Type":"ContainerDied","Data":"b1443f6a1ce8f6365763c31d8bcb62fd31fe1a0783f447264ee4483b1aaeb6fa"}
Jul 10 20:45:36.742461 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:36.742246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8" event={"ID":"a7460680-f9f8-4821-a9bc-6e2bbc52c670","Type":"ContainerStarted","Data":"446f276c625fb5131b970dd86b8bb26928861b2d8739d1a0fcb9008956074eae"}
Jul 10 20:45:37.400784 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.400740    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mt5wx"]
Jul 10 20:45:37.403799 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.403780    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:37.406520 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.406501    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-6n9b9\""
Jul 10 20:45:37.406629 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.406553    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 10 20:45:37.407695 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.407679    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 10 20:45:37.412543 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.412520    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mt5wx"]
Jul 10 20:45:37.478896 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.478810    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjkj4\" (UniqueName: \"kubernetes.io/projected/12b59a69-12bb-497d-be54-75d2cf6bd71c-kube-api-access-bjkj4\") pod \"cert-manager-webhook-6486ffd997-mt5wx\" (UID: \"12b59a69-12bb-497d-be54-75d2cf6bd71c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:37.478896 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.478871    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12b59a69-12bb-497d-be54-75d2cf6bd71c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mt5wx\" (UID: \"12b59a69-12bb-497d-be54-75d2cf6bd71c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:37.579866 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.579826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjkj4\" (UniqueName: \"kubernetes.io/projected/12b59a69-12bb-497d-be54-75d2cf6bd71c-kube-api-access-bjkj4\") pod \"cert-manager-webhook-6486ffd997-mt5wx\" (UID: \"12b59a69-12bb-497d-be54-75d2cf6bd71c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:37.580043 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.579886    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12b59a69-12bb-497d-be54-75d2cf6bd71c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mt5wx\" (UID: \"12b59a69-12bb-497d-be54-75d2cf6bd71c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:37.591013 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.590981    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12b59a69-12bb-497d-be54-75d2cf6bd71c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mt5wx\" (UID: \"12b59a69-12bb-497d-be54-75d2cf6bd71c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:37.591234 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.591207    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjkj4\" (UniqueName: \"kubernetes.io/projected/12b59a69-12bb-497d-be54-75d2cf6bd71c-kube-api-access-bjkj4\") pod \"cert-manager-webhook-6486ffd997-mt5wx\" (UID: \"12b59a69-12bb-497d-be54-75d2cf6bd71c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:37.727312 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.727260    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:37.884688 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:37.884657    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mt5wx"]
Jul 10 20:45:37.887350 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:45:37.887320    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12b59a69_12bb_497d_be54_75d2cf6bd71c.slice/crio-f329a5630a7cece5c7aa4ad4aea4a9df4dd0be0cd4486beeca26197543e4fa60 WatchSource:0}: Error finding container f329a5630a7cece5c7aa4ad4aea4a9df4dd0be0cd4486beeca26197543e4fa60: Status 404 returned error can't find the container with id f329a5630a7cece5c7aa4ad4aea4a9df4dd0be0cd4486beeca26197543e4fa60
Jul 10 20:45:38.757210 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:38.757172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx" event={"ID":"12b59a69-12bb-497d-be54-75d2cf6bd71c","Type":"ContainerStarted","Data":"f329a5630a7cece5c7aa4ad4aea4a9df4dd0be0cd4486beeca26197543e4fa60"}
Jul 10 20:45:39.763108 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:39.763074    2565 generic.go:356] "Generic (PLEG): container finished" podID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerID="29d6cd9ba01f4251ffab8c124fb448645766aa64a634a28a75393684636d12f2" exitCode=0
Jul 10 20:45:39.763469 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:39.763166    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8" event={"ID":"a7460680-f9f8-4821-a9bc-6e2bbc52c670","Type":"ContainerDied","Data":"29d6cd9ba01f4251ffab8c124fb448645766aa64a634a28a75393684636d12f2"}
Jul 10 20:45:40.769442 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:40.769403    2565 generic.go:356] "Generic (PLEG): container finished" podID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerID="1eee3bf7cc626aea72c0cb5f0484a8db6f2765d4c9afc3ecf2dd762eea8b6366" exitCode=0
Jul 10 20:45:40.769829 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:40.769450    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8" event={"ID":"a7460680-f9f8-4821-a9bc-6e2bbc52c670","Type":"ContainerDied","Data":"1eee3bf7cc626aea72c0cb5f0484a8db6f2765d4c9afc3ecf2dd762eea8b6366"}
Jul 10 20:45:41.935901 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:41.935865    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:42.017308 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.017259    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-bundle\") pod \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") "
Jul 10 20:45:42.017308 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.017310    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-util\") pod \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") "
Jul 10 20:45:42.017543 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.017344    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7tbz\" (UniqueName: \"kubernetes.io/projected/a7460680-f9f8-4821-a9bc-6e2bbc52c670-kube-api-access-k7tbz\") pod \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\" (UID: \"a7460680-f9f8-4821-a9bc-6e2bbc52c670\") "
Jul 10 20:45:42.017713 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.017687    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-bundle" (OuterVolumeSpecName: "bundle") pod "a7460680-f9f8-4821-a9bc-6e2bbc52c670" (UID: "a7460680-f9f8-4821-a9bc-6e2bbc52c670"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:45:42.019868 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.019843    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7460680-f9f8-4821-a9bc-6e2bbc52c670-kube-api-access-k7tbz" (OuterVolumeSpecName: "kube-api-access-k7tbz") pod "a7460680-f9f8-4821-a9bc-6e2bbc52c670" (UID: "a7460680-f9f8-4821-a9bc-6e2bbc52c670"). InnerVolumeSpecName "kube-api-access-k7tbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:45:42.087299 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.087253    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-util" (OuterVolumeSpecName: "util") pod "a7460680-f9f8-4821-a9bc-6e2bbc52c670" (UID: "a7460680-f9f8-4821-a9bc-6e2bbc52c670"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:45:42.118490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.118449    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:45:42.118490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.118482    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7460680-f9f8-4821-a9bc-6e2bbc52c670-util\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:45:42.118490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.118496    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7tbz\" (UniqueName: \"kubernetes.io/projected/a7460680-f9f8-4821-a9bc-6e2bbc52c670-kube-api-access-k7tbz\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:45:42.781228 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.781196    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8" event={"ID":"a7460680-f9f8-4821-a9bc-6e2bbc52c670","Type":"ContainerDied","Data":"446f276c625fb5131b970dd86b8bb26928861b2d8739d1a0fcb9008956074eae"}
Jul 10 20:45:42.781402 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.781243    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6qqx8"
Jul 10 20:45:42.781402 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.781248    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="446f276c625fb5131b970dd86b8bb26928861b2d8739d1a0fcb9008956074eae"
Jul 10 20:45:42.794357 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.794326    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"]
Jul 10 20:45:42.794818 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.794801    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerName="pull"
Jul 10 20:45:42.794901 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.794821    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerName="pull"
Jul 10 20:45:42.794901 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.794835    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerName="extract"
Jul 10 20:45:42.794901 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.794844    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerName="extract"
Jul 10 20:45:42.794901 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.794869    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerName="util"
Jul 10 20:45:42.794901 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.794877    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerName="util"
Jul 10 20:45:42.795073 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.794965    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7460680-f9f8-4821-a9bc-6e2bbc52c670" containerName="extract"
Jul 10 20:45:42.799281 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.799259    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"
Jul 10 20:45:42.802017 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.801996    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-d54z8\""
Jul 10 20:45:42.807857 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.807828    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"]
Jul 10 20:45:42.925837 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.925801    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb2b9608-e7f4-4b05-8b43-92678a09b840-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ckt2w\" (UID: \"cb2b9608-e7f4-4b05-8b43-92678a09b840\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"
Jul 10 20:45:42.926029 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:42.925977    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vtpq\" (UniqueName: \"kubernetes.io/projected/cb2b9608-e7f4-4b05-8b43-92678a09b840-kube-api-access-5vtpq\") pod \"cert-manager-cainjector-559cdd8c87-ckt2w\" (UID: \"cb2b9608-e7f4-4b05-8b43-92678a09b840\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"
Jul 10 20:45:43.027416 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.027376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vtpq\" (UniqueName: \"kubernetes.io/projected/cb2b9608-e7f4-4b05-8b43-92678a09b840-kube-api-access-5vtpq\") pod \"cert-manager-cainjector-559cdd8c87-ckt2w\" (UID: \"cb2b9608-e7f4-4b05-8b43-92678a09b840\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"
Jul 10 20:45:43.027853 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.027423    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb2b9608-e7f4-4b05-8b43-92678a09b840-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ckt2w\" (UID: \"cb2b9608-e7f4-4b05-8b43-92678a09b840\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"
Jul 10 20:45:43.036109 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.036040    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb2b9608-e7f4-4b05-8b43-92678a09b840-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ckt2w\" (UID: \"cb2b9608-e7f4-4b05-8b43-92678a09b840\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"
Jul 10 20:45:43.036109 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.036071    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vtpq\" (UniqueName: \"kubernetes.io/projected/cb2b9608-e7f4-4b05-8b43-92678a09b840-kube-api-access-5vtpq\") pod \"cert-manager-cainjector-559cdd8c87-ckt2w\" (UID: \"cb2b9608-e7f4-4b05-8b43-92678a09b840\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"
Jul 10 20:45:43.110678 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.110637    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"
Jul 10 20:45:43.248256 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.248146    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w"]
Jul 10 20:45:43.250319 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:45:43.250291    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb2b9608_e7f4_4b05_8b43_92678a09b840.slice/crio-523d3aee9e36e17f6a16844d0be63c37566de5d5206325859820b10d347e7057 WatchSource:0}: Error finding container 523d3aee9e36e17f6a16844d0be63c37566de5d5206325859820b10d347e7057: Status 404 returned error can't find the container with id 523d3aee9e36e17f6a16844d0be63c37566de5d5206325859820b10d347e7057
Jul 10 20:45:43.787444 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.787401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx" event={"ID":"12b59a69-12bb-497d-be54-75d2cf6bd71c","Type":"ContainerStarted","Data":"7d6161f8b607bcc855de817c78fabae93bf2ec2702e1488e88bbe6c4f747f191"}
Jul 10 20:45:43.787623 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.787459    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:43.788964 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.788939    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w" event={"ID":"cb2b9608-e7f4-4b05-8b43-92678a09b840","Type":"ContainerStarted","Data":"1d06da4814e6cc7fa350221e1ea224c5c57cd64dcdd96a41430e06e645f007ad"}
Jul 10 20:45:43.789051 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.788972    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w" event={"ID":"cb2b9608-e7f4-4b05-8b43-92678a09b840","Type":"ContainerStarted","Data":"523d3aee9e36e17f6a16844d0be63c37566de5d5206325859820b10d347e7057"}
Jul 10 20:45:43.806413 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.806370    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx" podStartSLOduration=1.525245096 podStartE2EDuration="6.806357102s" totalImagesPullingTime="5.281112006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:43.80496975 +0000 UTC m=+480.297765931" watchObservedRunningTime="2026-07-10 20:45:43.806357102 +0000 UTC m=+480.299153280"
Jul 10 20:45:43.824123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:43.824083    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ckt2w" podStartSLOduration=1.8240709640000001 podStartE2EDuration="1.824070964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:43.823205517 +0000 UTC m=+480.316001697" watchObservedRunningTime="2026-07-10 20:45:43.824070964 +0000 UTC m=+480.316867143"
Jul 10 20:45:46.681095 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.681055    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cl2jh"]
Jul 10 20:45:46.684483 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.684467    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cl2jh"
Jul 10 20:45:46.687209 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.687187    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dq5qv\""
Jul 10 20:45:46.696987 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.696956    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cl2jh"]
Jul 10 20:45:46.762552 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.762511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdqbh\" (UniqueName: \"kubernetes.io/projected/3b3d9f8a-6603-4be1-bd46-2c340f1e1f02-kube-api-access-pdqbh\") pod \"cert-manager-55b4d954d5-cl2jh\" (UID: \"3b3d9f8a-6603-4be1-bd46-2c340f1e1f02\") " pod="cert-manager/cert-manager-55b4d954d5-cl2jh"
Jul 10 20:45:46.762715 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.762580    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b3d9f8a-6603-4be1-bd46-2c340f1e1f02-bound-sa-token\") pod \"cert-manager-55b4d954d5-cl2jh\" (UID: \"3b3d9f8a-6603-4be1-bd46-2c340f1e1f02\") " pod="cert-manager/cert-manager-55b4d954d5-cl2jh"
Jul 10 20:45:46.863410 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.863376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdqbh\" (UniqueName: \"kubernetes.io/projected/3b3d9f8a-6603-4be1-bd46-2c340f1e1f02-kube-api-access-pdqbh\") pod \"cert-manager-55b4d954d5-cl2jh\" (UID: \"3b3d9f8a-6603-4be1-bd46-2c340f1e1f02\") " pod="cert-manager/cert-manager-55b4d954d5-cl2jh"
Jul 10 20:45:46.863410 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.863429    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b3d9f8a-6603-4be1-bd46-2c340f1e1f02-bound-sa-token\") pod \"cert-manager-55b4d954d5-cl2jh\" (UID: \"3b3d9f8a-6603-4be1-bd46-2c340f1e1f02\") " pod="cert-manager/cert-manager-55b4d954d5-cl2jh"
Jul 10 20:45:46.871876 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.871843    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b3d9f8a-6603-4be1-bd46-2c340f1e1f02-bound-sa-token\") pod \"cert-manager-55b4d954d5-cl2jh\" (UID: \"3b3d9f8a-6603-4be1-bd46-2c340f1e1f02\") " pod="cert-manager/cert-manager-55b4d954d5-cl2jh"
Jul 10 20:45:46.872059 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.872039    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdqbh\" (UniqueName: \"kubernetes.io/projected/3b3d9f8a-6603-4be1-bd46-2c340f1e1f02-kube-api-access-pdqbh\") pod \"cert-manager-55b4d954d5-cl2jh\" (UID: \"3b3d9f8a-6603-4be1-bd46-2c340f1e1f02\") " pod="cert-manager/cert-manager-55b4d954d5-cl2jh"
Jul 10 20:45:46.994679 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:46.994588    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cl2jh"
Jul 10 20:45:47.129982 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:47.129788    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cl2jh"]
Jul 10 20:45:47.132514 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:45:47.132478    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b3d9f8a_6603_4be1_bd46_2c340f1e1f02.slice/crio-e9bd74748a980c66bfec0130695d8e30940fd5a5bb3eec225ed49c18040d3687 WatchSource:0}: Error finding container e9bd74748a980c66bfec0130695d8e30940fd5a5bb3eec225ed49c18040d3687: Status 404 returned error can't find the container with id e9bd74748a980c66bfec0130695d8e30940fd5a5bb3eec225ed49c18040d3687
Jul 10 20:45:47.808236 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:47.808198    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cl2jh" event={"ID":"3b3d9f8a-6603-4be1-bd46-2c340f1e1f02","Type":"ContainerStarted","Data":"758d49d72ad2779c052e74a1f48a444fc8d72e39add92eae0b3d2652ec425344"}
Jul 10 20:45:47.808236 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:47.808251    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cl2jh" event={"ID":"3b3d9f8a-6603-4be1-bd46-2c340f1e1f02","Type":"ContainerStarted","Data":"e9bd74748a980c66bfec0130695d8e30940fd5a5bb3eec225ed49c18040d3687"}
Jul 10 20:45:47.826418 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:47.826359    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-cl2jh" podStartSLOduration=1.826345795 podStartE2EDuration="1.826345795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:47.825107487 +0000 UTC m=+484.317903665" watchObservedRunningTime="2026-07-10 20:45:47.826345795 +0000 UTC m=+484.319141974"
Jul 10 20:45:49.795701 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:49.795670    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5wx"
Jul 10 20:45:57.023289 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.023254    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"]
Jul 10 20:45:57.029143 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.029123    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.032028 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.032013    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-td5xw\""
Jul 10 20:45:57.032131 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.032034    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 10 20:45:57.032284 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.032266    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 10 20:45:57.038549 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.038525    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"]
Jul 10 20:45:57.150579 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.150523    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.150723 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.150590    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4mh8\" (UniqueName: \"kubernetes.io/projected/3a234868-1d02-4349-a32b-3d0d24320f54-kube-api-access-t4mh8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.150723 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.150617    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.251070 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.251036    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.251219 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.251075    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mh8\" (UniqueName: \"kubernetes.io/projected/3a234868-1d02-4349-a32b-3d0d24320f54-kube-api-access-t4mh8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.251219 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.251101    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.251389 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.251370    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.251522 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.251504    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.260137 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.260118    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4mh8\" (UniqueName: \"kubernetes.io/projected/3a234868-1d02-4349-a32b-3d0d24320f54-kube-api-access-t4mh8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.340522 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.340487    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:45:57.468822 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.468783    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"]
Jul 10 20:45:57.471030 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:45:57.471004    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a234868_1d02_4349_a32b_3d0d24320f54.slice/crio-e1512f712ae75bd109d85a60596eae9b8eb2e77aea9d5e4c2c5c2d8f992282b9 WatchSource:0}: Error finding container e1512f712ae75bd109d85a60596eae9b8eb2e77aea9d5e4c2c5c2d8f992282b9: Status 404 returned error can't find the container with id e1512f712ae75bd109d85a60596eae9b8eb2e77aea9d5e4c2c5c2d8f992282b9
Jul 10 20:45:57.856004 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.855971    2565 generic.go:356] "Generic (PLEG): container finished" podID="3a234868-1d02-4349-a32b-3d0d24320f54" containerID="5c158e02d9884e7fb36b47c8e7763b996ffe31f80798b87155fcb786ee3e0403" exitCode=0
Jul 10 20:45:57.856183 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.856047    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c" event={"ID":"3a234868-1d02-4349-a32b-3d0d24320f54","Type":"ContainerDied","Data":"5c158e02d9884e7fb36b47c8e7763b996ffe31f80798b87155fcb786ee3e0403"}
Jul 10 20:45:57.856183 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:57.856091    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c" event={"ID":"3a234868-1d02-4349-a32b-3d0d24320f54","Type":"ContainerStarted","Data":"e1512f712ae75bd109d85a60596eae9b8eb2e77aea9d5e4c2c5c2d8f992282b9"}
Jul 10 20:45:58.862190 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:58.862093    2565 generic.go:356] "Generic (PLEG): container finished" podID="3a234868-1d02-4349-a32b-3d0d24320f54" containerID="484c730c82412db647d3ab1b0332bb6b100edd35cc0b726791c0e919dcd085a2" exitCode=0
Jul 10 20:45:58.862538 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:58.862194    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c" event={"ID":"3a234868-1d02-4349-a32b-3d0d24320f54","Type":"ContainerDied","Data":"484c730c82412db647d3ab1b0332bb6b100edd35cc0b726791c0e919dcd085a2"}
Jul 10 20:45:59.869442 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:59.869408    2565 generic.go:356] "Generic (PLEG): container finished" podID="3a234868-1d02-4349-a32b-3d0d24320f54" containerID="9b49bab1cacd02185fd1beb75369f446df99ff9d8b25cf82ebb0c8a12b731d62" exitCode=0
Jul 10 20:45:59.869837 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:45:59.869485    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c" event={"ID":"3a234868-1d02-4349-a32b-3d0d24320f54","Type":"ContainerDied","Data":"9b49bab1cacd02185fd1beb75369f446df99ff9d8b25cf82ebb0c8a12b731d62"}
Jul 10 20:46:01.003348 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.003326    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:46:01.082633 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.082606    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-bundle\") pod \"3a234868-1d02-4349-a32b-3d0d24320f54\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") "
Jul 10 20:46:01.082758 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.082657    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-util\") pod \"3a234868-1d02-4349-a32b-3d0d24320f54\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") "
Jul 10 20:46:01.082758 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.082717    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4mh8\" (UniqueName: \"kubernetes.io/projected/3a234868-1d02-4349-a32b-3d0d24320f54-kube-api-access-t4mh8\") pod \"3a234868-1d02-4349-a32b-3d0d24320f54\" (UID: \"3a234868-1d02-4349-a32b-3d0d24320f54\") "
Jul 10 20:46:01.083475 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.083364    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-bundle" (OuterVolumeSpecName: "bundle") pod "3a234868-1d02-4349-a32b-3d0d24320f54" (UID: "3a234868-1d02-4349-a32b-3d0d24320f54"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:46:01.084971 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.084950    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a234868-1d02-4349-a32b-3d0d24320f54-kube-api-access-t4mh8" (OuterVolumeSpecName: "kube-api-access-t4mh8") pod "3a234868-1d02-4349-a32b-3d0d24320f54" (UID: "3a234868-1d02-4349-a32b-3d0d24320f54"). InnerVolumeSpecName "kube-api-access-t4mh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:46:01.090920 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.090896    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-util" (OuterVolumeSpecName: "util") pod "3a234868-1d02-4349-a32b-3d0d24320f54" (UID: "3a234868-1d02-4349-a32b-3d0d24320f54"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:46:01.183761 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.183680    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-util\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:46:01.183761 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.183713    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4mh8\" (UniqueName: \"kubernetes.io/projected/3a234868-1d02-4349-a32b-3d0d24320f54-kube-api-access-t4mh8\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:46:01.183761 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.183726    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a234868-1d02-4349-a32b-3d0d24320f54-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:46:01.880225 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.880190    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c" event={"ID":"3a234868-1d02-4349-a32b-3d0d24320f54","Type":"ContainerDied","Data":"e1512f712ae75bd109d85a60596eae9b8eb2e77aea9d5e4c2c5c2d8f992282b9"}
Jul 10 20:46:01.880391 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.880240    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1512f712ae75bd109d85a60596eae9b8eb2e77aea9d5e4c2c5c2d8f992282b9"
Jul 10 20:46:01.880391 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:01.880208    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjrm2c"
Jul 10 20:46:09.024335 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.024300    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"]
Jul 10 20:46:09.024711 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.024658    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a234868-1d02-4349-a32b-3d0d24320f54" containerName="extract"
Jul 10 20:46:09.024711 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.024669    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a234868-1d02-4349-a32b-3d0d24320f54" containerName="extract"
Jul 10 20:46:09.024711 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.024690    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a234868-1d02-4349-a32b-3d0d24320f54" containerName="util"
Jul 10 20:46:09.024711 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.024696    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a234868-1d02-4349-a32b-3d0d24320f54" containerName="util"
Jul 10 20:46:09.024711 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.024708    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a234868-1d02-4349-a32b-3d0d24320f54" containerName="pull"
Jul 10 20:46:09.024711 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.024713    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a234868-1d02-4349-a32b-3d0d24320f54" containerName="pull"
Jul 10 20:46:09.024947 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.024766    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a234868-1d02-4349-a32b-3d0d24320f54" containerName="extract"
Jul 10 20:46:09.027720 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.027702    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.030774 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.030752    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 20:46:09.031113 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.031097    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 20:46:09.031113 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.031107    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cndqb\""
Jul 10 20:46:09.031270 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.031199    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 20:46:09.031270 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.031230    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 20:46:09.046834 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.046810    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"]
Jul 10 20:46:09.153785 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.153756    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc7d9625-4d68-4299-8ad0-82d8779118d7-tmp\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.153950 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.153816    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc7d9625-4d68-4299-8ad0-82d8779118d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.153950 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.153834    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hz28\" (UniqueName: \"kubernetes.io/projected/cc7d9625-4d68-4299-8ad0-82d8779118d7-kube-api-access-7hz28\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.154055 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.153944    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc7d9625-4d68-4299-8ad0-82d8779118d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.154055 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.153986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc7d9625-4d68-4299-8ad0-82d8779118d7-manifests\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.255289 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.255260    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc7d9625-4d68-4299-8ad0-82d8779118d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.255289 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.255287    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hz28\" (UniqueName: \"kubernetes.io/projected/cc7d9625-4d68-4299-8ad0-82d8779118d7-kube-api-access-7hz28\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.255476 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.255328    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc7d9625-4d68-4299-8ad0-82d8779118d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.255610 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.255575    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc7d9625-4d68-4299-8ad0-82d8779118d7-manifests\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.255702 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.255628    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc7d9625-4d68-4299-8ad0-82d8779118d7-tmp\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.255884 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.255859    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc7d9625-4d68-4299-8ad0-82d8779118d7-manifests\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.257769 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.257742    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc7d9625-4d68-4299-8ad0-82d8779118d7-tmp\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.257984 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.257968    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc7d9625-4d68-4299-8ad0-82d8779118d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.258029 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.258000    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc7d9625-4d68-4299-8ad0-82d8779118d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.263044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.263024    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hz28\" (UniqueName: \"kubernetes.io/projected/cc7d9625-4d68-4299-8ad0-82d8779118d7-kube-api-access-7hz28\") pod \"opendatahub-operator-controller-manager-6558bffdbc-b2zgh\" (UID: \"cc7d9625-4d68-4299-8ad0-82d8779118d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.338014 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.337986    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:09.483401 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.483369    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"]
Jul 10 20:46:09.487411 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:46:09.487374    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc7d9625_4d68_4299_8ad0_82d8779118d7.slice/crio-e4617aed2599b911b4e650819b99694ece0d3f9b98042b1831c6a2dcac8c3167 WatchSource:0}: Error finding container e4617aed2599b911b4e650819b99694ece0d3f9b98042b1831c6a2dcac8c3167: Status 404 returned error can't find the container with id e4617aed2599b911b4e650819b99694ece0d3f9b98042b1831c6a2dcac8c3167
Jul 10 20:46:09.611951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.611878    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"]
Jul 10 20:46:09.616272 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.616251    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.619213 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.619191    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 20:46:09.619333 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.619194    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 20:46:09.619333 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.619304    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 20:46:09.619504 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.619488    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:46:09.619594 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.619529    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 20:46:09.619663 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.619626    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zr82z\""
Jul 10 20:46:09.629130 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.629102    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"]
Jul 10 20:46:09.759499 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.759464    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2392acb8-240a-42ab-8c97-c282c0c64489-manager-config\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.759499 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.759498    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsllf\" (UniqueName: \"kubernetes.io/projected/2392acb8-240a-42ab-8c97-c282c0c64489-kube-api-access-wsllf\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.759713 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.759520    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2392acb8-240a-42ab-8c97-c282c0c64489-metrics-cert\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.759713 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.759599    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2392acb8-240a-42ab-8c97-c282c0c64489-cert\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.860710 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.860674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2392acb8-240a-42ab-8c97-c282c0c64489-manager-config\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.860710 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.860711    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsllf\" (UniqueName: \"kubernetes.io/projected/2392acb8-240a-42ab-8c97-c282c0c64489-kube-api-access-wsllf\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.860976 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.860733    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2392acb8-240a-42ab-8c97-c282c0c64489-metrics-cert\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.860976 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.860760    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2392acb8-240a-42ab-8c97-c282c0c64489-cert\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.861399 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.861375    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2392acb8-240a-42ab-8c97-c282c0c64489-manager-config\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.863323 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.863280    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2392acb8-240a-42ab-8c97-c282c0c64489-cert\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.863381 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.863328    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2392acb8-240a-42ab-8c97-c282c0c64489-metrics-cert\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.872003 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.871979    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsllf\" (UniqueName: \"kubernetes.io/projected/2392acb8-240a-42ab-8c97-c282c0c64489-kube-api-access-wsllf\") pod \"lws-controller-manager-66b48cb7bf-qgn2z\" (UID: \"2392acb8-240a-42ab-8c97-c282c0c64489\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:09.924184 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.924151    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh" event={"ID":"cc7d9625-4d68-4299-8ad0-82d8779118d7","Type":"ContainerStarted","Data":"e4617aed2599b911b4e650819b99694ece0d3f9b98042b1831c6a2dcac8c3167"}
Jul 10 20:46:09.925342 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:09.925316    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:10.076393 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:10.076285    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"]
Jul 10 20:46:10.079640 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:46:10.079610    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2392acb8_240a_42ab_8c97_c282c0c64489.slice/crio-357d195467e036dee1f76cf1079f819e3c171619d604120db54612c53087fb30 WatchSource:0}: Error finding container 357d195467e036dee1f76cf1079f819e3c171619d604120db54612c53087fb30: Status 404 returned error can't find the container with id 357d195467e036dee1f76cf1079f819e3c171619d604120db54612c53087fb30
Jul 10 20:46:10.934232 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:10.934119    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z" event={"ID":"2392acb8-240a-42ab-8c97-c282c0c64489","Type":"ContainerStarted","Data":"357d195467e036dee1f76cf1079f819e3c171619d604120db54612c53087fb30"}
Jul 10 20:46:13.950551 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:13.950518    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh" event={"ID":"cc7d9625-4d68-4299-8ad0-82d8779118d7","Type":"ContainerStarted","Data":"6cea7a54e3abe3aaf9ca50814b81311fa0022211cb556d096a760f30f9077120"}
Jul 10 20:46:13.956472 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:13.956448    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:13.956606 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:13.956486    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z" event={"ID":"2392acb8-240a-42ab-8c97-c282c0c64489","Type":"ContainerStarted","Data":"3bcedc484863e9786ba967032da272b923db65079b67aef1e0daa90178204f33"}
Jul 10 20:46:13.993156 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:13.993108    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z" podStartSLOduration=2.196905305 podStartE2EDuration="4.993089062s" totalImagesPullingTime="2.796183757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:46:13.992177941 +0000 UTC m=+510.484974120" watchObservedRunningTime="2026-07-10 20:46:13.993089062 +0000 UTC m=+510.485885244"
Jul 10 20:46:14.958215 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:14.958180    2565 generic.go:356] "Generic (PLEG): container finished" podID="cc7d9625-4d68-4299-8ad0-82d8779118d7" containerID="6cea7a54e3abe3aaf9ca50814b81311fa0022211cb556d096a760f30f9077120" exitCode=0
Jul 10 20:46:14.958669 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:14.958266    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh" event={"ID":"cc7d9625-4d68-4299-8ad0-82d8779118d7","Type":"ContainerDied","Data":"6cea7a54e3abe3aaf9ca50814b81311fa0022211cb556d096a760f30f9077120"}
Jul 10 20:46:15.966395 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:15.966354    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh" event={"ID":"cc7d9625-4d68-4299-8ad0-82d8779118d7","Type":"ContainerStarted","Data":"ed578a7f5a70ad5375021dd63da5762889edb2c7afdf699f330e752a5740058c"}
Jul 10 20:46:15.966787 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:15.966466    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:15.988301 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:15.988256    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh" podStartSLOduration=3.13956469 podStartE2EDuration="7.988243351s" totalImagesPullingTime="3.694703555s" totalInitContainerRuntime="1.153975106s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:46:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:46:15.986467374 +0000 UTC m=+512.479263553" watchObservedRunningTime="2026-07-10 20:46:15.988243351 +0000 UTC m=+512.481039530"
Jul 10 20:46:24.960143 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:24.960109    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-qgn2z"
Jul 10 20:46:26.974056 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:26.974023    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-b2zgh"
Jul 10 20:46:29.255607 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.255568    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"]
Jul 10 20:46:29.272117 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.272087    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.272386 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.272089    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"]
Jul 10 20:46:29.275128 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.275103    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 20:46:29.275239 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.275163    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dmhn5\""
Jul 10 20:46:29.275239 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.275184    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 20:46:29.276345 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.276314    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 20:46:29.276345 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.276344    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 10 20:46:29.323428 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.323402    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c98ed95a-26fc-4dc8-880b-5c379f0d19b6-tls-certs\") pod \"kube-auth-proxy-86d9dbc65b-5wdn8\" (UID: \"c98ed95a-26fc-4dc8-880b-5c379f0d19b6\") " pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.323585 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.323459    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c98ed95a-26fc-4dc8-880b-5c379f0d19b6-tmp\") pod \"kube-auth-proxy-86d9dbc65b-5wdn8\" (UID: \"c98ed95a-26fc-4dc8-880b-5c379f0d19b6\") " pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.323585 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.323479    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz4m7\" (UniqueName: \"kubernetes.io/projected/c98ed95a-26fc-4dc8-880b-5c379f0d19b6-kube-api-access-sz4m7\") pod \"kube-auth-proxy-86d9dbc65b-5wdn8\" (UID: \"c98ed95a-26fc-4dc8-880b-5c379f0d19b6\") " pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.424083 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.424052    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c98ed95a-26fc-4dc8-880b-5c379f0d19b6-tls-certs\") pod \"kube-auth-proxy-86d9dbc65b-5wdn8\" (UID: \"c98ed95a-26fc-4dc8-880b-5c379f0d19b6\") " pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.424232 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.424111    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c98ed95a-26fc-4dc8-880b-5c379f0d19b6-tmp\") pod \"kube-auth-proxy-86d9dbc65b-5wdn8\" (UID: \"c98ed95a-26fc-4dc8-880b-5c379f0d19b6\") " pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.424232 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.424137    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4m7\" (UniqueName: \"kubernetes.io/projected/c98ed95a-26fc-4dc8-880b-5c379f0d19b6-kube-api-access-sz4m7\") pod \"kube-auth-proxy-86d9dbc65b-5wdn8\" (UID: \"c98ed95a-26fc-4dc8-880b-5c379f0d19b6\") " pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.426587 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.426531    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c98ed95a-26fc-4dc8-880b-5c379f0d19b6-tmp\") pod \"kube-auth-proxy-86d9dbc65b-5wdn8\" (UID: \"c98ed95a-26fc-4dc8-880b-5c379f0d19b6\") " pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.426771 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.426755    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c98ed95a-26fc-4dc8-880b-5c379f0d19b6-tls-certs\") pod \"kube-auth-proxy-86d9dbc65b-5wdn8\" (UID: \"c98ed95a-26fc-4dc8-880b-5c379f0d19b6\") " pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.498614 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.498582    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz4m7\" (UniqueName: \"kubernetes.io/projected/c98ed95a-26fc-4dc8-880b-5c379f0d19b6-kube-api-access-sz4m7\") pod \"kube-auth-proxy-86d9dbc65b-5wdn8\" (UID: \"c98ed95a-26fc-4dc8-880b-5c379f0d19b6\") " pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.583982 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.583953    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"
Jul 10 20:46:29.719245 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:29.719221    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8"]
Jul 10 20:46:29.720724 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:46:29.720703    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98ed95a_26fc_4dc8_880b_5c379f0d19b6.slice/crio-ac163e0282b8187eefce944687092546506ff67640639ef78cad9b654d43dca5 WatchSource:0}: Error finding container ac163e0282b8187eefce944687092546506ff67640639ef78cad9b654d43dca5: Status 404 returned error can't find the container with id ac163e0282b8187eefce944687092546506ff67640639ef78cad9b654d43dca5
Jul 10 20:46:30.035071 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:30.034988    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8" event={"ID":"c98ed95a-26fc-4dc8-880b-5c379f0d19b6","Type":"ContainerStarted","Data":"ac163e0282b8187eefce944687092546506ff67640639ef78cad9b654d43dca5"}
Jul 10 20:46:33.053782 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:33.053745    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8" event={"ID":"c98ed95a-26fc-4dc8-880b-5c379f0d19b6","Type":"ContainerStarted","Data":"53fa86958d58ed0cfc5f7ab59fdd96aabfe119f15b65b0064f131d0e3e77ef14"}
Jul 10 20:46:33.070926 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:46:33.070880    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86d9dbc65b-5wdn8" podStartSLOduration=1.373831293 podStartE2EDuration="4.07086457s" totalImagesPullingTime="2.697033277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:46:33.06960261 +0000 UTC m=+529.562398789" watchObservedRunningTime="2026-07-10 20:46:33.07086457 +0000 UTC m=+529.563660749"
Jul 10 20:47:44.131135 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:44.131104    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:47:44.133631 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:44.132996    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:47:44.136367 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:44.136352    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:47:44.138049 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:44.138025    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:47:46.856695 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.856653    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"]
Jul 10 20:47:46.860393 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.860370    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:46.863227 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.863191    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 20:47:46.863354 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.863325    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 20:47:46.863416 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.863404    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dwf2s\""
Jul 10 20:47:46.867233 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.867212    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"]
Jul 10 20:47:46.921205 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.921149    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:46.921388 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.921238    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkgzz\" (UniqueName: \"kubernetes.io/projected/5112aa1a-f16f-49c5-875a-3653fd6a3a45-kube-api-access-wkgzz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:46.921388 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.921307    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:46.961461 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.961428    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"]
Jul 10 20:47:46.965005 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.964988    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:46.973798 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:46.973772    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"]
Jul 10 20:47:47.022368 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.022332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:47.022549 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.022392    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkgzz\" (UniqueName: \"kubernetes.io/projected/5112aa1a-f16f-49c5-875a-3653fd6a3a45-kube-api-access-wkgzz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:47.022645 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.022629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:47.022864 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.022839    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:47.022997 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.022970    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:47.031483 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.031461    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkgzz\" (UniqueName: \"kubernetes.io/projected/5112aa1a-f16f-49c5-875a-3653fd6a3a45-kube-api-access-wkgzz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:47.066224 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.066195    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"]
Jul 10 20:47:47.069877 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.069861    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.077985 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.077963    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"]
Jul 10 20:47:47.123864 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.123778    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.123864 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.123843    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr4tf\" (UniqueName: \"kubernetes.io/projected/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-kube-api-access-xr4tf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.124029 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.123873    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.124029 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.123896    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.124029 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.123960    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.124029 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.124006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfzd9\" (UniqueName: \"kubernetes.io/projected/1528f9d5-c452-4658-ab5d-defe6aebe5b6-kube-api-access-rfzd9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.171657 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.171624    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:47.224440 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.224412    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.224592 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.224484    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfzd9\" (UniqueName: \"kubernetes.io/projected/1528f9d5-c452-4658-ab5d-defe6aebe5b6-kube-api-access-rfzd9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.224592 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.224527    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.224726 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.224599    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr4tf\" (UniqueName: \"kubernetes.io/projected/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-kube-api-access-xr4tf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.224726 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.224629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.224726 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.224663    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.225053 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.224908    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.225053 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.225000    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.225258 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.225239    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.225317 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.225261    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.235130 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.235007    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfzd9\" (UniqueName: \"kubernetes.io/projected/1528f9d5-c452-4658-ab5d-defe6aebe5b6-kube-api-access-rfzd9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.235646 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.235610    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr4tf\" (UniqueName: \"kubernetes.io/projected/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-kube-api-access-xr4tf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.274712 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.274682    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:47.311466 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.311436    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"]
Jul 10 20:47:47.314656 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:47:47.314620    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5112aa1a_f16f_49c5_875a_3653fd6a3a45.slice/crio-8335a6d3b874d832bf9c49cbe8fdbeef8949865f4d2f09386c8a634f45197aef WatchSource:0}: Error finding container 8335a6d3b874d832bf9c49cbe8fdbeef8949865f4d2f09386c8a634f45197aef: Status 404 returned error can't find the container with id 8335a6d3b874d832bf9c49cbe8fdbeef8949865f4d2f09386c8a634f45197aef
Jul 10 20:47:47.380544 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.380475    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:47.415256 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.415216    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd" event={"ID":"5112aa1a-f16f-49c5-875a-3653fd6a3a45","Type":"ContainerStarted","Data":"e38fad3b2f07c5d1ef6ba8188c775bcf268a7c205e958ee52fd41bcf08054117"}
Jul 10 20:47:47.415416 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.415287    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd" event={"ID":"5112aa1a-f16f-49c5-875a-3653fd6a3a45","Type":"ContainerStarted","Data":"8335a6d3b874d832bf9c49cbe8fdbeef8949865f4d2f09386c8a634f45197aef"}
Jul 10 20:47:47.424585 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.424532    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"]
Jul 10 20:47:47.470751 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.470723    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"]
Jul 10 20:47:47.474329 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.474308    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.487308 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.487276    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"]
Jul 10 20:47:47.526775 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.526746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.526936 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.526888    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8nps\" (UniqueName: \"kubernetes.io/projected/2d6eda7a-3957-49dd-baff-d74160f88abd-kube-api-access-x8nps\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.526936 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.526928    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.539241 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.539216    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"]
Jul 10 20:47:47.540755 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:47:47.540725    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1528f9d5_c452_4658_ab5d_defe6aebe5b6.slice/crio-fd5843a76d614a6c6054b6694680261e00eaa1411b52010e88f0d96ad36f9202 WatchSource:0}: Error finding container fd5843a76d614a6c6054b6694680261e00eaa1411b52010e88f0d96ad36f9202: Status 404 returned error can't find the container with id fd5843a76d614a6c6054b6694680261e00eaa1411b52010e88f0d96ad36f9202
Jul 10 20:47:47.627887 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.627858    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.628007 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.627988    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8nps\" (UniqueName: \"kubernetes.io/projected/2d6eda7a-3957-49dd-baff-d74160f88abd-kube-api-access-x8nps\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.628074 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.628025    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.628182 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.628163    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.628324 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.628306    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.641104 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.641052    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8nps\" (UniqueName: \"kubernetes.io/projected/2d6eda7a-3957-49dd-baff-d74160f88abd-kube-api-access-x8nps\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.796074 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.796043    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:47.927414 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:47.927385    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"]
Jul 10 20:47:47.929410 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:47:47.929380    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6eda7a_3957_49dd_baff_d74160f88abd.slice/crio-92fb03918d6655e9da791e059e3af21b8469e6ee7a2d6b69581e559294ce0c6b WatchSource:0}: Error finding container 92fb03918d6655e9da791e059e3af21b8469e6ee7a2d6b69581e559294ce0c6b: Status 404 returned error can't find the container with id 92fb03918d6655e9da791e059e3af21b8469e6ee7a2d6b69581e559294ce0c6b
Jul 10 20:47:48.422069 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.422026    2565 generic.go:356] "Generic (PLEG): container finished" podID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerID="e38fad3b2f07c5d1ef6ba8188c775bcf268a7c205e958ee52fd41bcf08054117" exitCode=0
Jul 10 20:47:48.422248 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.422117    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd" event={"ID":"5112aa1a-f16f-49c5-875a-3653fd6a3a45","Type":"ContainerDied","Data":"e38fad3b2f07c5d1ef6ba8188c775bcf268a7c205e958ee52fd41bcf08054117"}
Jul 10 20:47:48.423792 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.423766    2565 generic.go:356] "Generic (PLEG): container finished" podID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerID="dedfa4ffcf8810e3756f209bf4dbee08d3545eed594cf859d83267daa69f608a" exitCode=0
Jul 10 20:47:48.423912 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.423851    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96" event={"ID":"1528f9d5-c452-4658-ab5d-defe6aebe5b6","Type":"ContainerDied","Data":"dedfa4ffcf8810e3756f209bf4dbee08d3545eed594cf859d83267daa69f608a"}
Jul 10 20:47:48.423912 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.423901    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96" event={"ID":"1528f9d5-c452-4658-ab5d-defe6aebe5b6","Type":"ContainerStarted","Data":"fd5843a76d614a6c6054b6694680261e00eaa1411b52010e88f0d96ad36f9202"}
Jul 10 20:47:48.425469 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.425376    2565 generic.go:356] "Generic (PLEG): container finished" podID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerID="21f1d13ae68aba4858da81766ee8c330cbbf3e206bc0221dc29c54b2f5c7aae0" exitCode=0
Jul 10 20:47:48.425469 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.425446    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8" event={"ID":"2d6eda7a-3957-49dd-baff-d74160f88abd","Type":"ContainerDied","Data":"21f1d13ae68aba4858da81766ee8c330cbbf3e206bc0221dc29c54b2f5c7aae0"}
Jul 10 20:47:48.425639 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.425477    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8" event={"ID":"2d6eda7a-3957-49dd-baff-d74160f88abd","Type":"ContainerStarted","Data":"92fb03918d6655e9da791e059e3af21b8469e6ee7a2d6b69581e559294ce0c6b"}
Jul 10 20:47:48.427164 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.427144    2565 generic.go:356] "Generic (PLEG): container finished" podID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerID="741e9cf3dfd12cbaabbb7198de93da3fcf7e4125c858764e2dc96ad399bcd6b2" exitCode=0
Jul 10 20:47:48.427241 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.427173    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5" event={"ID":"c8205fd1-e2ec-418c-9011-ba1acfbed1fa","Type":"ContainerDied","Data":"741e9cf3dfd12cbaabbb7198de93da3fcf7e4125c858764e2dc96ad399bcd6b2"}
Jul 10 20:47:48.427241 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:48.427207    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5" event={"ID":"c8205fd1-e2ec-418c-9011-ba1acfbed1fa","Type":"ContainerStarted","Data":"971eef18f883256552e30c160bc7c879d4cb431ca5938a545faeb1d8ec26d0d6"}
Jul 10 20:47:49.434993 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:49.434962    2565 generic.go:356] "Generic (PLEG): container finished" podID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerID="013ce3122bdf14532fe5b86fd8d50cff6a9b15c7fa5e1025087072ca42a1d7ae" exitCode=0
Jul 10 20:47:49.435351 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:49.435046    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5" event={"ID":"c8205fd1-e2ec-418c-9011-ba1acfbed1fa","Type":"ContainerDied","Data":"013ce3122bdf14532fe5b86fd8d50cff6a9b15c7fa5e1025087072ca42a1d7ae"}
Jul 10 20:47:50.442090 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:50.442008    2565 generic.go:356] "Generic (PLEG): container finished" podID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerID="7f217c714027e1844c73a1027a08f338c51e0045e54f9f82f5fb0bc6c7703e2e" exitCode=0
Jul 10 20:47:50.442090 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:50.442074    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd" event={"ID":"5112aa1a-f16f-49c5-875a-3653fd6a3a45","Type":"ContainerDied","Data":"7f217c714027e1844c73a1027a08f338c51e0045e54f9f82f5fb0bc6c7703e2e"}
Jul 10 20:47:50.443957 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:50.443932    2565 generic.go:356] "Generic (PLEG): container finished" podID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerID="f4cdad8e81869606e85d1c270b71f4cb53ce442e1e25cdb8bc23ce8bbe2acb24" exitCode=0
Jul 10 20:47:50.444038 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:50.444015    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96" event={"ID":"1528f9d5-c452-4658-ab5d-defe6aebe5b6","Type":"ContainerDied","Data":"f4cdad8e81869606e85d1c270b71f4cb53ce442e1e25cdb8bc23ce8bbe2acb24"}
Jul 10 20:47:50.445985 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:50.445955    2565 generic.go:356] "Generic (PLEG): container finished" podID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerID="fc91b9f85397e92c13868811ea81c3d83cb998f4d1798a1d7e3b807c42ebfda9" exitCode=0
Jul 10 20:47:50.446068 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:50.445984    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8" event={"ID":"2d6eda7a-3957-49dd-baff-d74160f88abd","Type":"ContainerDied","Data":"fc91b9f85397e92c13868811ea81c3d83cb998f4d1798a1d7e3b807c42ebfda9"}
Jul 10 20:47:50.448425 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:50.448402    2565 generic.go:356] "Generic (PLEG): container finished" podID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerID="be57ebf2e4e7cc93015d553b01f2bb8fd7adb188815c456fa777d4fbc6a6101c" exitCode=0
Jul 10 20:47:50.448508 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:50.448477    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5" event={"ID":"c8205fd1-e2ec-418c-9011-ba1acfbed1fa","Type":"ContainerDied","Data":"be57ebf2e4e7cc93015d553b01f2bb8fd7adb188815c456fa777d4fbc6a6101c"}
Jul 10 20:47:51.457247 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.457158    2565 generic.go:356] "Generic (PLEG): container finished" podID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerID="2a46ce3c8e465b51d6e0504b05d00c041c2ed1a847db05ea81246f35e412fb2c" exitCode=0
Jul 10 20:47:51.457659 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.457237    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd" event={"ID":"5112aa1a-f16f-49c5-875a-3653fd6a3a45","Type":"ContainerDied","Data":"2a46ce3c8e465b51d6e0504b05d00c041c2ed1a847db05ea81246f35e412fb2c"}
Jul 10 20:47:51.466448 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.466414    2565 generic.go:356] "Generic (PLEG): container finished" podID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerID="acc9ebe27bd95a1f125263ef94efd7daa919d8e0232a241e590f501e5ec408a9" exitCode=0
Jul 10 20:47:51.466642 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.466501    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96" event={"ID":"1528f9d5-c452-4658-ab5d-defe6aebe5b6","Type":"ContainerDied","Data":"acc9ebe27bd95a1f125263ef94efd7daa919d8e0232a241e590f501e5ec408a9"}
Jul 10 20:47:51.468912 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.468875    2565 generic.go:356] "Generic (PLEG): container finished" podID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerID="91c4f1b3cfaa6d3d31fdc7194624082976526957612e99da40d5b3ed1c5be7f3" exitCode=0
Jul 10 20:47:51.469029 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.468969    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8" event={"ID":"2d6eda7a-3957-49dd-baff-d74160f88abd","Type":"ContainerDied","Data":"91c4f1b3cfaa6d3d31fdc7194624082976526957612e99da40d5b3ed1c5be7f3"}
Jul 10 20:47:51.601972 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.601949    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:51.665969 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.665936    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr4tf\" (UniqueName: \"kubernetes.io/projected/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-kube-api-access-xr4tf\") pod \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") "
Jul 10 20:47:51.666141 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.665999    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-bundle\") pod \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") "
Jul 10 20:47:51.666141 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.666058    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-util\") pod \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\" (UID: \"c8205fd1-e2ec-418c-9011-ba1acfbed1fa\") "
Jul 10 20:47:51.666480 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.666454    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-bundle" (OuterVolumeSpecName: "bundle") pod "c8205fd1-e2ec-418c-9011-ba1acfbed1fa" (UID: "c8205fd1-e2ec-418c-9011-ba1acfbed1fa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:47:51.668210 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.668186    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-kube-api-access-xr4tf" (OuterVolumeSpecName: "kube-api-access-xr4tf") pod "c8205fd1-e2ec-418c-9011-ba1acfbed1fa" (UID: "c8205fd1-e2ec-418c-9011-ba1acfbed1fa"). InnerVolumeSpecName "kube-api-access-xr4tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:47:51.671444 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.671408    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-util" (OuterVolumeSpecName: "util") pod "c8205fd1-e2ec-418c-9011-ba1acfbed1fa" (UID: "c8205fd1-e2ec-418c-9011-ba1acfbed1fa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:47:51.766798 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.766728    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr4tf\" (UniqueName: \"kubernetes.io/projected/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-kube-api-access-xr4tf\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:51.766798 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.766753    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:51.766798 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:51.766762    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8205fd1-e2ec-418c-9011-ba1acfbed1fa-util\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:52.476143 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.476103    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5"
Jul 10 20:47:52.476518 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.476147    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5" event={"ID":"c8205fd1-e2ec-418c-9011-ba1acfbed1fa","Type":"ContainerDied","Data":"971eef18f883256552e30c160bc7c879d4cb431ca5938a545faeb1d8ec26d0d6"}
Jul 10 20:47:52.476518 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.476195    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="971eef18f883256552e30c160bc7c879d4cb431ca5938a545faeb1d8ec26d0d6"
Jul 10 20:47:52.633691 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.633670    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:52.657041 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.657021    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:52.660979 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.660957    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:52.675369 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.675326    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-util\") pod \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") "
Jul 10 20:47:52.675507 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.675366    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-bundle\") pod \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") "
Jul 10 20:47:52.675507 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.675429    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfzd9\" (UniqueName: \"kubernetes.io/projected/1528f9d5-c452-4658-ab5d-defe6aebe5b6-kube-api-access-rfzd9\") pod \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") "
Jul 10 20:47:52.675507 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.675476    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-util\") pod \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\" (UID: \"1528f9d5-c452-4658-ab5d-defe6aebe5b6\") "
Jul 10 20:47:52.675507 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.675492    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-util\") pod \"2d6eda7a-3957-49dd-baff-d74160f88abd\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") "
Jul 10 20:47:52.675728 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.675534    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-bundle\") pod \"2d6eda7a-3957-49dd-baff-d74160f88abd\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") "
Jul 10 20:47:52.675728 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.675580    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkgzz\" (UniqueName: \"kubernetes.io/projected/5112aa1a-f16f-49c5-875a-3653fd6a3a45-kube-api-access-wkgzz\") pod \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") "
Jul 10 20:47:52.675728 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.675615    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8nps\" (UniqueName: \"kubernetes.io/projected/2d6eda7a-3957-49dd-baff-d74160f88abd-kube-api-access-x8nps\") pod \"2d6eda7a-3957-49dd-baff-d74160f88abd\" (UID: \"2d6eda7a-3957-49dd-baff-d74160f88abd\") "
Jul 10 20:47:52.675728 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.675642    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-bundle\") pod \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\" (UID: \"5112aa1a-f16f-49c5-875a-3653fd6a3a45\") "
Jul 10 20:47:52.676259 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.676230    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-bundle" (OuterVolumeSpecName: "bundle") pod "1528f9d5-c452-4658-ab5d-defe6aebe5b6" (UID: "1528f9d5-c452-4658-ab5d-defe6aebe5b6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:47:52.676530 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.676491    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-bundle" (OuterVolumeSpecName: "bundle") pod "5112aa1a-f16f-49c5-875a-3653fd6a3a45" (UID: "5112aa1a-f16f-49c5-875a-3653fd6a3a45"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:47:52.677820 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.677769    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-bundle" (OuterVolumeSpecName: "bundle") pod "2d6eda7a-3957-49dd-baff-d74160f88abd" (UID: "2d6eda7a-3957-49dd-baff-d74160f88abd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:47:52.680168 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.680142    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1528f9d5-c452-4658-ab5d-defe6aebe5b6-kube-api-access-rfzd9" (OuterVolumeSpecName: "kube-api-access-rfzd9") pod "1528f9d5-c452-4658-ab5d-defe6aebe5b6" (UID: "1528f9d5-c452-4658-ab5d-defe6aebe5b6"). InnerVolumeSpecName "kube-api-access-rfzd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:47:52.682108 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.682081    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5112aa1a-f16f-49c5-875a-3653fd6a3a45-kube-api-access-wkgzz" (OuterVolumeSpecName: "kube-api-access-wkgzz") pod "5112aa1a-f16f-49c5-875a-3653fd6a3a45" (UID: "5112aa1a-f16f-49c5-875a-3653fd6a3a45"). InnerVolumeSpecName "kube-api-access-wkgzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:47:52.683709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.683667    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-util" (OuterVolumeSpecName: "util") pod "5112aa1a-f16f-49c5-875a-3653fd6a3a45" (UID: "5112aa1a-f16f-49c5-875a-3653fd6a3a45"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:47:52.683709 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.683663    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d6eda7a-3957-49dd-baff-d74160f88abd-kube-api-access-x8nps" (OuterVolumeSpecName: "kube-api-access-x8nps") pod "2d6eda7a-3957-49dd-baff-d74160f88abd" (UID: "2d6eda7a-3957-49dd-baff-d74160f88abd"). InnerVolumeSpecName "kube-api-access-x8nps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:47:52.684804 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.684762    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-util" (OuterVolumeSpecName: "util") pod "1528f9d5-c452-4658-ab5d-defe6aebe5b6" (UID: "1528f9d5-c452-4658-ab5d-defe6aebe5b6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:47:52.685569 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.685542    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-util" (OuterVolumeSpecName: "util") pod "2d6eda7a-3957-49dd-baff-d74160f88abd" (UID: "2d6eda7a-3957-49dd-baff-d74160f88abd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:47:52.777044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.776960    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-util\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:52.777044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.776989    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:52.777044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.776998    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfzd9\" (UniqueName: \"kubernetes.io/projected/1528f9d5-c452-4658-ab5d-defe6aebe5b6-kube-api-access-rfzd9\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:52.777044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.777007    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1528f9d5-c452-4658-ab5d-defe6aebe5b6-util\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:52.777044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.777016    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-util\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:52.777044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.777023    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d6eda7a-3957-49dd-baff-d74160f88abd-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:52.777044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.777031    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkgzz\" (UniqueName: \"kubernetes.io/projected/5112aa1a-f16f-49c5-875a-3653fd6a3a45-kube-api-access-wkgzz\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:52.777044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.777040    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8nps\" (UniqueName: \"kubernetes.io/projected/2d6eda7a-3957-49dd-baff-d74160f88abd-kube-api-access-x8nps\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:52.777044 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:52.777049    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5112aa1a-f16f-49c5-875a-3653fd6a3a45-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:47:53.483281 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:53.483253    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd"
Jul 10 20:47:53.483704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:53.483255    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd" event={"ID":"5112aa1a-f16f-49c5-875a-3653fd6a3a45","Type":"ContainerDied","Data":"8335a6d3b874d832bf9c49cbe8fdbeef8949865f4d2f09386c8a634f45197aef"}
Jul 10 20:47:53.483704 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:53.483392    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8335a6d3b874d832bf9c49cbe8fdbeef8949865f4d2f09386c8a634f45197aef"
Jul 10 20:47:53.485496 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:53.485477    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96"
Jul 10 20:47:53.485496 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:53.485484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96" event={"ID":"1528f9d5-c452-4658-ab5d-defe6aebe5b6","Type":"ContainerDied","Data":"fd5843a76d614a6c6054b6694680261e00eaa1411b52010e88f0d96ad36f9202"}
Jul 10 20:47:53.485725 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:53.485519    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd5843a76d614a6c6054b6694680261e00eaa1411b52010e88f0d96ad36f9202"
Jul 10 20:47:53.487520 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:53.487494    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8" event={"ID":"2d6eda7a-3957-49dd-baff-d74160f88abd","Type":"ContainerDied","Data":"92fb03918d6655e9da791e059e3af21b8469e6ee7a2d6b69581e559294ce0c6b"}
Jul 10 20:47:53.487638 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:53.487524    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8"
Jul 10 20:47:53.487638 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:47:53.487534    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92fb03918d6655e9da791e059e3af21b8469e6ee7a2d6b69581e559294ce0c6b"
Jul 10 20:48:08.394536 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.394498    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c6955dd9d-cdmlh"]
Jul 10 20:48:08.395043 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395020    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerName="extract"
Jul 10 20:48:08.395123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395048    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerName="extract"
Jul 10 20:48:08.395123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395063    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerName="util"
Jul 10 20:48:08.395123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395072    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerName="util"
Jul 10 20:48:08.395123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395083    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerName="util"
Jul 10 20:48:08.395123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395091    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerName="util"
Jul 10 20:48:08.395123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395103    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerName="pull"
Jul 10 20:48:08.395123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395111    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerName="pull"
Jul 10 20:48:08.395123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395119    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerName="extract"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395129    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerName="extract"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395143    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerName="pull"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395151    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerName="pull"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395161    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerName="pull"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395169    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerName="pull"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395186    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerName="util"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395194    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerName="util"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395210    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerName="util"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395218    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerName="util"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395234    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerName="extract"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395242    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerName="extract"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395253    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerName="extract"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395261    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerName="extract"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395277    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerName="pull"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395284    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerName="pull"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395380    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8205fd1-e2ec-418c-9011-ba1acfbed1fa" containerName="extract"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395391    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5112aa1a-f16f-49c5-875a-3653fd6a3a45" containerName="extract"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395406    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1528f9d5-c452-4658-ab5d-defe6aebe5b6" containerName="extract"
Jul 10 20:48:08.395490 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.395414    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d6eda7a-3957-49dd-baff-d74160f88abd" containerName="extract"
Jul 10 20:48:08.398793 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.398770    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.412005 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.411979    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6955dd9d-cdmlh"]
Jul 10 20:48:08.506739 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.506707    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-console-config\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.506926 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.506774    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-trusted-ca-bundle\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.506926 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.506838    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b41a85dc-ea87-4401-a07d-c68cdb5231de-console-oauth-config\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.506926 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.506871    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-service-ca\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.506926 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.506894    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-oauth-serving-cert\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.506926 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.506909    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5gz4\" (UniqueName: \"kubernetes.io/projected/b41a85dc-ea87-4401-a07d-c68cdb5231de-kube-api-access-t5gz4\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.507109 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.506933    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b41a85dc-ea87-4401-a07d-c68cdb5231de-console-serving-cert\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.607825 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.607792    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-service-ca\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.607825 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.607826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-oauth-serving-cert\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.608016 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.607841    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5gz4\" (UniqueName: \"kubernetes.io/projected/b41a85dc-ea87-4401-a07d-c68cdb5231de-kube-api-access-t5gz4\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.608016 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.607957    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b41a85dc-ea87-4401-a07d-c68cdb5231de-console-serving-cert\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.608016 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.608003    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-console-config\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.608211 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.608084    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-trusted-ca-bundle\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.608211 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.608109    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b41a85dc-ea87-4401-a07d-c68cdb5231de-console-oauth-config\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.608577 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.608540    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-oauth-serving-cert\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.608700 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.608591    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-service-ca\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.608700 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.608694    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-console-config\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.608930 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.608908    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b41a85dc-ea87-4401-a07d-c68cdb5231de-trusted-ca-bundle\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.610539 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.610497    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b41a85dc-ea87-4401-a07d-c68cdb5231de-console-serving-cert\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.610643 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.610616    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b41a85dc-ea87-4401-a07d-c68cdb5231de-console-oauth-config\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.616326 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.616310    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5gz4\" (UniqueName: \"kubernetes.io/projected/b41a85dc-ea87-4401-a07d-c68cdb5231de-kube-api-access-t5gz4\") pod \"console-c6955dd9d-cdmlh\" (UID: \"b41a85dc-ea87-4401-a07d-c68cdb5231de\") " pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.709244 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.709171    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:08.861085 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:08.861062    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6955dd9d-cdmlh"]
Jul 10 20:48:08.862615 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:48:08.862585    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41a85dc_ea87_4401_a07d_c68cdb5231de.slice/crio-583762ef25ee78f5d6ced06242589b3cd5fc8246df8ae64ab0c08a6567a2c7de WatchSource:0}: Error finding container 583762ef25ee78f5d6ced06242589b3cd5fc8246df8ae64ab0c08a6567a2c7de: Status 404 returned error can't find the container with id 583762ef25ee78f5d6ced06242589b3cd5fc8246df8ae64ab0c08a6567a2c7de
Jul 10 20:48:09.569967 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:09.569930    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6955dd9d-cdmlh" event={"ID":"b41a85dc-ea87-4401-a07d-c68cdb5231de","Type":"ContainerStarted","Data":"dad066c84c91268b5e05652ed7fbd8a49e990defef3c0399f69deabb2091f7a3"}
Jul 10 20:48:09.569967 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:09.569978    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6955dd9d-cdmlh" event={"ID":"b41a85dc-ea87-4401-a07d-c68cdb5231de","Type":"ContainerStarted","Data":"583762ef25ee78f5d6ced06242589b3cd5fc8246df8ae64ab0c08a6567a2c7de"}
Jul 10 20:48:09.589882 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:09.589840    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c6955dd9d-cdmlh" podStartSLOduration=1.589825824 podStartE2EDuration="1.589825824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:48:09.588043092 +0000 UTC m=+626.080839273" watchObservedRunningTime="2026-07-10 20:48:09.589825824 +0000 UTC m=+626.082622002"
Jul 10 20:48:18.709548 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:18.709510    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:18.710034 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:18.709614    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:18.715095 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:18.715071    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:19.624865 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:19.624838    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c6955dd9d-cdmlh"
Jul 10 20:48:19.669205 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:19.669177    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bf64dbcc-nftsc"]
Jul 10 20:48:30.321585 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.321520    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"]
Jul 10 20:48:30.326639 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.326616    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"
Jul 10 20:48:30.329639 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.329615    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hj2g2\""
Jul 10 20:48:30.330137 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.330119    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 20:48:30.330895 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.330876    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 20:48:30.338017 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.337991    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"]
Jul 10 20:48:30.405689 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.405652    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8st4\" (UniqueName: \"kubernetes.io/projected/09abe612-6421-49d1-be23-6b9fefea04e4-kube-api-access-v8st4\") pod \"limitador-operator-controller-manager-859d7b78bd-wnsj8\" (UID: \"09abe612-6421-49d1-be23-6b9fefea04e4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"
Jul 10 20:48:30.506919 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.506875    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8st4\" (UniqueName: \"kubernetes.io/projected/09abe612-6421-49d1-be23-6b9fefea04e4-kube-api-access-v8st4\") pod \"limitador-operator-controller-manager-859d7b78bd-wnsj8\" (UID: \"09abe612-6421-49d1-be23-6b9fefea04e4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"
Jul 10 20:48:30.515454 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.515420    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8st4\" (UniqueName: \"kubernetes.io/projected/09abe612-6421-49d1-be23-6b9fefea04e4-kube-api-access-v8st4\") pod \"limitador-operator-controller-manager-859d7b78bd-wnsj8\" (UID: \"09abe612-6421-49d1-be23-6b9fefea04e4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"
Jul 10 20:48:30.644593 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.644453    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"
Jul 10 20:48:30.778752 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.778720    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"]
Jul 10 20:48:30.781406 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:48:30.781378    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09abe612_6421_49d1_be23_6b9fefea04e4.slice/crio-6bdade494bf561afdc6da22ffb80dad63badccbfad5532ab5bd147e207f1a123 WatchSource:0}: Error finding container 6bdade494bf561afdc6da22ffb80dad63badccbfad5532ab5bd147e207f1a123: Status 404 returned error can't find the container with id 6bdade494bf561afdc6da22ffb80dad63badccbfad5532ab5bd147e207f1a123
Jul 10 20:48:30.783699 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:30.783681    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 20:48:31.683656 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:31.683621    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8" event={"ID":"09abe612-6421-49d1-be23-6b9fefea04e4","Type":"ContainerStarted","Data":"6bdade494bf561afdc6da22ffb80dad63badccbfad5532ab5bd147e207f1a123"}
Jul 10 20:48:33.695836 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:33.695800    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8" event={"ID":"09abe612-6421-49d1-be23-6b9fefea04e4","Type":"ContainerStarted","Data":"01d858a0285bf0c6396752a569434844ad0215c3d1a8948f20ded4ae06c40f5c"}
Jul 10 20:48:33.696177 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:33.695907    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"
Jul 10 20:48:33.712577 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:33.712515    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8" podStartSLOduration=1.8494167209999999 podStartE2EDuration="3.712501129s" totalImagesPullingTime="1.863084408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:48:33.710368296 +0000 UTC m=+650.203164476" watchObservedRunningTime="2026-07-10 20:48:33.712501129 +0000 UTC m=+650.205297308"
Jul 10 20:48:44.691817 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:44.691751    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bf64dbcc-nftsc" podUID="11590b47-f7fa-45a2-aa30-10dee877e842" containerName="console" containerID="cri-o://5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793" gracePeriod=15
Jul 10 20:48:44.702674 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:44.702649    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wnsj8"
Jul 10 20:48:44.936720 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:44.936698    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:48:45.034951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.034881    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-serving-cert\") pod \"11590b47-f7fa-45a2-aa30-10dee877e842\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") "
Jul 10 20:48:45.034951 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.034912    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-trusted-ca-bundle\") pod \"11590b47-f7fa-45a2-aa30-10dee877e842\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") "
Jul 10 20:48:45.035123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.034963    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-oauth-serving-cert\") pod \"11590b47-f7fa-45a2-aa30-10dee877e842\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") "
Jul 10 20:48:45.035123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.034988    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxk7q\" (UniqueName: \"kubernetes.io/projected/11590b47-f7fa-45a2-aa30-10dee877e842-kube-api-access-wxk7q\") pod \"11590b47-f7fa-45a2-aa30-10dee877e842\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") "
Jul 10 20:48:45.035123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.035030    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-oauth-config\") pod \"11590b47-f7fa-45a2-aa30-10dee877e842\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") "
Jul 10 20:48:45.035123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.035046    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-console-config\") pod \"11590b47-f7fa-45a2-aa30-10dee877e842\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") "
Jul 10 20:48:45.035123 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.035087    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-service-ca\") pod \"11590b47-f7fa-45a2-aa30-10dee877e842\" (UID: \"11590b47-f7fa-45a2-aa30-10dee877e842\") "
Jul 10 20:48:45.035418 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.035401    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "11590b47-f7fa-45a2-aa30-10dee877e842" (UID: "11590b47-f7fa-45a2-aa30-10dee877e842"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:48:45.035877 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.035528    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "11590b47-f7fa-45a2-aa30-10dee877e842" (UID: "11590b47-f7fa-45a2-aa30-10dee877e842"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:48:45.035877 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.035646    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-service-ca" (OuterVolumeSpecName: "service-ca") pod "11590b47-f7fa-45a2-aa30-10dee877e842" (UID: "11590b47-f7fa-45a2-aa30-10dee877e842"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:48:45.035877 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.035682    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-console-config" (OuterVolumeSpecName: "console-config") pod "11590b47-f7fa-45a2-aa30-10dee877e842" (UID: "11590b47-f7fa-45a2-aa30-10dee877e842"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:48:45.037119 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.037098    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "11590b47-f7fa-45a2-aa30-10dee877e842" (UID: "11590b47-f7fa-45a2-aa30-10dee877e842"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:48:45.037262 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.037240    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "11590b47-f7fa-45a2-aa30-10dee877e842" (UID: "11590b47-f7fa-45a2-aa30-10dee877e842"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:48:45.037311 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.037256    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11590b47-f7fa-45a2-aa30-10dee877e842-kube-api-access-wxk7q" (OuterVolumeSpecName: "kube-api-access-wxk7q") pod "11590b47-f7fa-45a2-aa30-10dee877e842" (UID: "11590b47-f7fa-45a2-aa30-10dee877e842"). InnerVolumeSpecName "kube-api-access-wxk7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:48:45.135849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.135816    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-oauth-serving-cert\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:48:45.135849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.135845    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxk7q\" (UniqueName: \"kubernetes.io/projected/11590b47-f7fa-45a2-aa30-10dee877e842-kube-api-access-wxk7q\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:48:45.135849 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.135855    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-oauth-config\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:48:45.136041 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.135865    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-console-config\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:48:45.136041 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.135873    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:48:45.136041 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.135881    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11590b47-f7fa-45a2-aa30-10dee877e842-console-serving-cert\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:48:45.136041 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.135889    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11590b47-f7fa-45a2-aa30-10dee877e842-trusted-ca-bundle\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:48:45.758440 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.758404    2565 generic.go:356] "Generic (PLEG): container finished" podID="11590b47-f7fa-45a2-aa30-10dee877e842" containerID="5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793" exitCode=0
Jul 10 20:48:45.758872 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.758448    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf64dbcc-nftsc"
Jul 10 20:48:45.758872 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.758469    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf64dbcc-nftsc" event={"ID":"11590b47-f7fa-45a2-aa30-10dee877e842","Type":"ContainerDied","Data":"5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793"}
Jul 10 20:48:45.758872 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.758510    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf64dbcc-nftsc" event={"ID":"11590b47-f7fa-45a2-aa30-10dee877e842","Type":"ContainerDied","Data":"7670d3143f8c68f96052b1557fa038d5e1c86c069ac21a6616d177680a83edaf"}
Jul 10 20:48:45.758872 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.758529    2565 scope.go:117] "RemoveContainer" containerID="5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793"
Jul 10 20:48:45.767943 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.767926    2565 scope.go:117] "RemoveContainer" containerID="5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793"
Jul 10 20:48:45.768236 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:48:45.768218    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793\": container with ID starting with 5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793 not found: ID does not exist" containerID="5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793"
Jul 10 20:48:45.768287 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.768246    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793"} err="failed to get container status \"5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793\": rpc error: code = NotFound desc = could not find container \"5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793\": container with ID starting with 5c6259cf83b60fdb72ef51de757b1bc4cbb06d4b6c7a2c1b073a4b724abcb793 not found: ID does not exist"
Jul 10 20:48:45.779406 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.779383    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bf64dbcc-nftsc"]
Jul 10 20:48:45.785500 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.785481    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bf64dbcc-nftsc"]
Jul 10 20:48:45.957263 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:48:45.957230    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11590b47-f7fa-45a2-aa30-10dee877e842" path="/var/lib/kubelet/pods/11590b47-f7fa-45a2-aa30-10dee877e842/volumes"
Jul 10 20:49:25.263926 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.263887    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wknwh"]
Jul 10 20:49:25.264451 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.264430    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11590b47-f7fa-45a2-aa30-10dee877e842" containerName="console"
Jul 10 20:49:25.264532 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.264456    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="11590b47-f7fa-45a2-aa30-10dee877e842" containerName="console"
Jul 10 20:49:25.264621 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.264599    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="11590b47-f7fa-45a2-aa30-10dee877e842" containerName="console"
Jul 10 20:49:25.267775 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.267754    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wknwh"
Jul 10 20:49:25.270383 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.270365    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-szm5d\""
Jul 10 20:49:25.274259 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.274236    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wknwh"]
Jul 10 20:49:25.277012 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.276986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmccn\" (UniqueName: \"kubernetes.io/projected/480cf0fa-64a7-4107-a0e3-80224d4e94f3-kube-api-access-qmccn\") pod \"authorino-75d556bd77-wknwh\" (UID: \"480cf0fa-64a7-4107-a0e3-80224d4e94f3\") " pod="kuadrant-system/authorino-75d556bd77-wknwh"
Jul 10 20:49:25.377931 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.377900    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmccn\" (UniqueName: \"kubernetes.io/projected/480cf0fa-64a7-4107-a0e3-80224d4e94f3-kube-api-access-qmccn\") pod \"authorino-75d556bd77-wknwh\" (UID: \"480cf0fa-64a7-4107-a0e3-80224d4e94f3\") " pod="kuadrant-system/authorino-75d556bd77-wknwh"
Jul 10 20:49:25.386512 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.386489    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmccn\" (UniqueName: \"kubernetes.io/projected/480cf0fa-64a7-4107-a0e3-80224d4e94f3-kube-api-access-qmccn\") pod \"authorino-75d556bd77-wknwh\" (UID: \"480cf0fa-64a7-4107-a0e3-80224d4e94f3\") " pod="kuadrant-system/authorino-75d556bd77-wknwh"
Jul 10 20:49:25.578131 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.578102    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wknwh"
Jul 10 20:49:25.706921 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.706895    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wknwh"]
Jul 10 20:49:25.709005 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:49:25.708971    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod480cf0fa_64a7_4107_a0e3_80224d4e94f3.slice/crio-03710cc5b2be36510f040cfb5dda23fc81ae52e89c8e60600a28041e58ea1b31 WatchSource:0}: Error finding container 03710cc5b2be36510f040cfb5dda23fc81ae52e89c8e60600a28041e58ea1b31: Status 404 returned error can't find the container with id 03710cc5b2be36510f040cfb5dda23fc81ae52e89c8e60600a28041e58ea1b31
Jul 10 20:49:25.980023 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:25.979938    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wknwh" event={"ID":"480cf0fa-64a7-4107-a0e3-80224d4e94f3","Type":"ContainerStarted","Data":"03710cc5b2be36510f040cfb5dda23fc81ae52e89c8e60600a28041e58ea1b31"}
Jul 10 20:49:29.000625 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:29.000520    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wknwh" event={"ID":"480cf0fa-64a7-4107-a0e3-80224d4e94f3","Type":"ContainerStarted","Data":"0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17"}
Jul 10 20:49:29.017068 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:29.017020    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wknwh" podStartSLOduration=1.1427813900000001 podStartE2EDuration="4.017005278s" totalImagesPullingTime="2.874223888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:49:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:49:29.015668617 +0000 UTC m=+705.508464808" watchObservedRunningTime="2026-07-10 20:49:29.017005278 +0000 UTC m=+705.509801456"
Jul 10 20:49:53.439882 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.439839    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ngt66"]
Jul 10 20:49:53.444751 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.444727    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ngt66"
Jul 10 20:49:53.448534 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.448503    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ngt66"]
Jul 10 20:49:53.496022 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.495990    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr8gm\" (UniqueName: \"kubernetes.io/projected/8c9a298d-e83f-4e9e-a789-b8a9a8b409b3-kube-api-access-dr8gm\") pod \"authorino-5d4b67c446-ngt66\" (UID: \"8c9a298d-e83f-4e9e-a789-b8a9a8b409b3\") " pod="kuadrant-system/authorino-5d4b67c446-ngt66"
Jul 10 20:49:53.596992 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.596956    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr8gm\" (UniqueName: \"kubernetes.io/projected/8c9a298d-e83f-4e9e-a789-b8a9a8b409b3-kube-api-access-dr8gm\") pod \"authorino-5d4b67c446-ngt66\" (UID: \"8c9a298d-e83f-4e9e-a789-b8a9a8b409b3\") " pod="kuadrant-system/authorino-5d4b67c446-ngt66"
Jul 10 20:49:53.606821 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.606787    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr8gm\" (UniqueName: \"kubernetes.io/projected/8c9a298d-e83f-4e9e-a789-b8a9a8b409b3-kube-api-access-dr8gm\") pod \"authorino-5d4b67c446-ngt66\" (UID: \"8c9a298d-e83f-4e9e-a789-b8a9a8b409b3\") " pod="kuadrant-system/authorino-5d4b67c446-ngt66"
Jul 10 20:49:53.696957 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.696878    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ngt66"]
Jul 10 20:49:53.697133 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.697121    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ngt66"
Jul 10 20:49:53.720584 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.720538    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-785c98fdb4-c8mjf"]
Jul 10 20:49:53.724195 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.724171    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-785c98fdb4-c8mjf"
Jul 10 20:49:53.729393 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.729361    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-785c98fdb4-c8mjf"]
Jul 10 20:49:53.799291 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.799255    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5p7l\" (UniqueName: \"kubernetes.io/projected/693edfec-b84e-4521-8d5a-50d6f53bd268-kube-api-access-v5p7l\") pod \"authorino-785c98fdb4-c8mjf\" (UID: \"693edfec-b84e-4521-8d5a-50d6f53bd268\") " pod="kuadrant-system/authorino-785c98fdb4-c8mjf"
Jul 10 20:49:53.833775 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.833749    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ngt66"]
Jul 10 20:49:53.836360 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:49:53.836323    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c9a298d_e83f_4e9e_a789_b8a9a8b409b3.slice/crio-4f027bd9be59ebe358ae9e5d4927e5884f1dd1e19aaf16fc4c87ce7364925226 WatchSource:0}: Error finding container 4f027bd9be59ebe358ae9e5d4927e5884f1dd1e19aaf16fc4c87ce7364925226: Status 404 returned error can't find the container with id 4f027bd9be59ebe358ae9e5d4927e5884f1dd1e19aaf16fc4c87ce7364925226
Jul 10 20:49:53.895642 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.895606    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-785c98fdb4-c8mjf"]
Jul 10 20:49:53.895879 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:49:53.895861    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-v5p7l], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-785c98fdb4-c8mjf" podUID="693edfec-b84e-4521-8d5a-50d6f53bd268"
Jul 10 20:49:53.900626 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.900603    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5p7l\" (UniqueName: \"kubernetes.io/projected/693edfec-b84e-4521-8d5a-50d6f53bd268-kube-api-access-v5p7l\") pod \"authorino-785c98fdb4-c8mjf\" (UID: \"693edfec-b84e-4521-8d5a-50d6f53bd268\") " pod="kuadrant-system/authorino-785c98fdb4-c8mjf"
Jul 10 20:49:53.911605 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.911585    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5p7l\" (UniqueName: \"kubernetes.io/projected/693edfec-b84e-4521-8d5a-50d6f53bd268-kube-api-access-v5p7l\") pod \"authorino-785c98fdb4-c8mjf\" (UID: \"693edfec-b84e-4521-8d5a-50d6f53bd268\") " pod="kuadrant-system/authorino-785c98fdb4-c8mjf"
Jul 10 20:49:53.922915 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.922889    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8c5976d96-lqprb"]
Jul 10 20:49:53.926386 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.926370    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:49:53.929072 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.929053    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 10 20:49:53.933184 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:53.933162    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8c5976d96-lqprb"]
Jul 10 20:49:54.001421 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.001279    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c69tt\" (UniqueName: \"kubernetes.io/projected/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-kube-api-access-c69tt\") pod \"authorino-8c5976d96-lqprb\" (UID: \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\") " pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:49:54.001421 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.001318    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-tls-cert\") pod \"authorino-8c5976d96-lqprb\" (UID: \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\") " pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:49:54.102227 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.102188    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c69tt\" (UniqueName: \"kubernetes.io/projected/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-kube-api-access-c69tt\") pod \"authorino-8c5976d96-lqprb\" (UID: \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\") " pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:49:54.102227 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.102230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-tls-cert\") pod \"authorino-8c5976d96-lqprb\" (UID: \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\") " pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:49:54.104809 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.104788    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-tls-cert\") pod \"authorino-8c5976d96-lqprb\" (UID: \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\") " pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:49:54.110258 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.110233    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c69tt\" (UniqueName: \"kubernetes.io/projected/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-kube-api-access-c69tt\") pod \"authorino-8c5976d96-lqprb\" (UID: \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\") " pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:49:54.128889 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.128857    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ngt66" event={"ID":"8c9a298d-e83f-4e9e-a789-b8a9a8b409b3","Type":"ContainerStarted","Data":"4f027bd9be59ebe358ae9e5d4927e5884f1dd1e19aaf16fc4c87ce7364925226"}
Jul 10 20:49:54.129047 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.128943    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-785c98fdb4-c8mjf"
Jul 10 20:49:54.135616 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.135594    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-785c98fdb4-c8mjf"
Jul 10 20:49:54.203383 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.203348    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5p7l\" (UniqueName: \"kubernetes.io/projected/693edfec-b84e-4521-8d5a-50d6f53bd268-kube-api-access-v5p7l\") pod \"693edfec-b84e-4521-8d5a-50d6f53bd268\" (UID: \"693edfec-b84e-4521-8d5a-50d6f53bd268\") "
Jul 10 20:49:54.205469 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.205439    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/693edfec-b84e-4521-8d5a-50d6f53bd268-kube-api-access-v5p7l" (OuterVolumeSpecName: "kube-api-access-v5p7l") pod "693edfec-b84e-4521-8d5a-50d6f53bd268" (UID: "693edfec-b84e-4521-8d5a-50d6f53bd268"). InnerVolumeSpecName "kube-api-access-v5p7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:49:54.236590 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.236532    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:49:54.304939 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.304912    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5p7l\" (UniqueName: \"kubernetes.io/projected/693edfec-b84e-4521-8d5a-50d6f53bd268-kube-api-access-v5p7l\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:49:54.368830 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:54.368798    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8c5976d96-lqprb"]
Jul 10 20:49:54.370187 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:49:54.370163    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad6747e3_acdc_4ae6_b6bf_37d2ffb5b6eb.slice/crio-3537133b937c15f27d3e7cce72e2ceeb64f90be8b99c852252d9d2cf52e79852 WatchSource:0}: Error finding container 3537133b937c15f27d3e7cce72e2ceeb64f90be8b99c852252d9d2cf52e79852: Status 404 returned error can't find the container with id 3537133b937c15f27d3e7cce72e2ceeb64f90be8b99c852252d9d2cf52e79852
Jul 10 20:49:55.141947 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.141904    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c5976d96-lqprb" event={"ID":"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb","Type":"ContainerStarted","Data":"8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d"}
Jul 10 20:49:55.142385 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.141955    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c5976d96-lqprb" event={"ID":"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb","Type":"ContainerStarted","Data":"3537133b937c15f27d3e7cce72e2ceeb64f90be8b99c852252d9d2cf52e79852"}
Jul 10 20:49:55.143406 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.143386    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-785c98fdb4-c8mjf"
Jul 10 20:49:55.143406 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.143397    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ngt66" event={"ID":"8c9a298d-e83f-4e9e-a789-b8a9a8b409b3","Type":"ContainerStarted","Data":"ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9"}
Jul 10 20:49:55.143605 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.143420    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-ngt66" podUID="8c9a298d-e83f-4e9e-a789-b8a9a8b409b3" containerName="authorino" containerID="cri-o://ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9" gracePeriod=30
Jul 10 20:49:55.160922 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.160876    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8c5976d96-lqprb" podStartSLOduration=1.885854562 podStartE2EDuration="2.160861209s" totalImagesPullingTime="275.006647ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:49:55.159651431 +0000 UTC m=+731.652447608" watchObservedRunningTime="2026-07-10 20:49:55.160861209 +0000 UTC m=+731.653657387"
Jul 10 20:49:55.174546 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.174492    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-ngt66" podStartSLOduration=1.526328201 podStartE2EDuration="2.174475936s" totalImagesPullingTime="648.147735ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:49:55.173094648 +0000 UTC m=+731.665890828" watchObservedRunningTime="2026-07-10 20:49:55.174475936 +0000 UTC m=+731.667272116"
Jul 10 20:49:55.189982 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.189940    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wknwh"]
Jul 10 20:49:55.190173 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.190149    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wknwh" podUID="480cf0fa-64a7-4107-a0e3-80224d4e94f3" containerName="authorino" containerID="cri-o://0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17" gracePeriod=30
Jul 10 20:49:55.204289 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.204262    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-785c98fdb4-c8mjf"]
Jul 10 20:49:55.210961 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.210931    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-785c98fdb4-c8mjf"]
Jul 10 20:49:55.442380 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.442351    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ngt66"
Jul 10 20:49:55.451841 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.451822    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wknwh"
Jul 10 20:49:55.515487 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.515449    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmccn\" (UniqueName: \"kubernetes.io/projected/480cf0fa-64a7-4107-a0e3-80224d4e94f3-kube-api-access-qmccn\") pod \"480cf0fa-64a7-4107-a0e3-80224d4e94f3\" (UID: \"480cf0fa-64a7-4107-a0e3-80224d4e94f3\") "
Jul 10 20:49:55.515487 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.515488    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr8gm\" (UniqueName: \"kubernetes.io/projected/8c9a298d-e83f-4e9e-a789-b8a9a8b409b3-kube-api-access-dr8gm\") pod \"8c9a298d-e83f-4e9e-a789-b8a9a8b409b3\" (UID: \"8c9a298d-e83f-4e9e-a789-b8a9a8b409b3\") "
Jul 10 20:49:55.517715 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.517692    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9a298d-e83f-4e9e-a789-b8a9a8b409b3-kube-api-access-dr8gm" (OuterVolumeSpecName: "kube-api-access-dr8gm") pod "8c9a298d-e83f-4e9e-a789-b8a9a8b409b3" (UID: "8c9a298d-e83f-4e9e-a789-b8a9a8b409b3"). InnerVolumeSpecName "kube-api-access-dr8gm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:49:55.517801 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.517752    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/480cf0fa-64a7-4107-a0e3-80224d4e94f3-kube-api-access-qmccn" (OuterVolumeSpecName: "kube-api-access-qmccn") pod "480cf0fa-64a7-4107-a0e3-80224d4e94f3" (UID: "480cf0fa-64a7-4107-a0e3-80224d4e94f3"). InnerVolumeSpecName "kube-api-access-qmccn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:49:55.616373 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.616345    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmccn\" (UniqueName: \"kubernetes.io/projected/480cf0fa-64a7-4107-a0e3-80224d4e94f3-kube-api-access-qmccn\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:49:55.616474 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.616376    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr8gm\" (UniqueName: \"kubernetes.io/projected/8c9a298d-e83f-4e9e-a789-b8a9a8b409b3-kube-api-access-dr8gm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:49:55.958144 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:55.958110    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="693edfec-b84e-4521-8d5a-50d6f53bd268" path="/var/lib/kubelet/pods/693edfec-b84e-4521-8d5a-50d6f53bd268/volumes"
Jul 10 20:49:56.150434 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.150398    2565 generic.go:356] "Generic (PLEG): container finished" podID="8c9a298d-e83f-4e9e-a789-b8a9a8b409b3" containerID="ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9" exitCode=0
Jul 10 20:49:56.150930 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.150446    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ngt66"
Jul 10 20:49:56.150930 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.150484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ngt66" event={"ID":"8c9a298d-e83f-4e9e-a789-b8a9a8b409b3","Type":"ContainerDied","Data":"ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9"}
Jul 10 20:49:56.150930 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.150529    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ngt66" event={"ID":"8c9a298d-e83f-4e9e-a789-b8a9a8b409b3","Type":"ContainerDied","Data":"4f027bd9be59ebe358ae9e5d4927e5884f1dd1e19aaf16fc4c87ce7364925226"}
Jul 10 20:49:56.150930 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.150550    2565 scope.go:117] "RemoveContainer" containerID="ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9"
Jul 10 20:49:56.151932 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.151913    2565 generic.go:356] "Generic (PLEG): container finished" podID="480cf0fa-64a7-4107-a0e3-80224d4e94f3" containerID="0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17" exitCode=0
Jul 10 20:49:56.152011 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.151956    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wknwh"
Jul 10 20:49:56.152011 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.151984    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wknwh" event={"ID":"480cf0fa-64a7-4107-a0e3-80224d4e94f3","Type":"ContainerDied","Data":"0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17"}
Jul 10 20:49:56.152109 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.152012    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wknwh" event={"ID":"480cf0fa-64a7-4107-a0e3-80224d4e94f3","Type":"ContainerDied","Data":"03710cc5b2be36510f040cfb5dda23fc81ae52e89c8e60600a28041e58ea1b31"}
Jul 10 20:49:56.159656 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.159618    2565 scope.go:117] "RemoveContainer" containerID="ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9"
Jul 10 20:49:56.159901 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:49:56.159881    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9\": container with ID starting with ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9 not found: ID does not exist" containerID="ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9"
Jul 10 20:49:56.159977 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.159913    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9"} err="failed to get container status \"ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9\": rpc error: code = NotFound desc = could not find container \"ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9\": container with ID starting with ca8d96248554e640ff4de39086eae3afecb3f3f91c8c2ecaf064171dfb5ba4d9 not found: ID does not exist"
Jul 10 20:49:56.159977 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.159934    2565 scope.go:117] "RemoveContainer" containerID="0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17"
Jul 10 20:49:56.168035 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.168013    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wknwh"]
Jul 10 20:49:56.169127 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.169112    2565 scope.go:117] "RemoveContainer" containerID="0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17"
Jul 10 20:49:56.169376 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:49:56.169352    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17\": container with ID starting with 0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17 not found: ID does not exist" containerID="0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17"
Jul 10 20:49:56.169420 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.169384    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17"} err="failed to get container status \"0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17\": rpc error: code = NotFound desc = could not find container \"0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17\": container with ID starting with 0c1fc8e39ac0307ae91706b03b9aa7110439d1f3dd2025005babb8fd95786c17 not found: ID does not exist"
Jul 10 20:49:56.172747 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.172728    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wknwh"]
Jul 10 20:49:56.183282 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.183263    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ngt66"]
Jul 10 20:49:56.186389 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:56.186366    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ngt66"]
Jul 10 20:49:57.958003 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:57.957971    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="480cf0fa-64a7-4107-a0e3-80224d4e94f3" path="/var/lib/kubelet/pods/480cf0fa-64a7-4107-a0e3-80224d4e94f3/volumes"
Jul 10 20:49:57.958344 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:49:57.958278    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9a298d-e83f-4e9e-a789-b8a9a8b409b3" path="/var/lib/kubelet/pods/8c9a298d-e83f-4e9e-a789-b8a9a8b409b3/volumes"
Jul 10 20:51:24.054482 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.054447    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"]
Jul 10 20:51:24.054922 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.054830    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="480cf0fa-64a7-4107-a0e3-80224d4e94f3" containerName="authorino"
Jul 10 20:51:24.054922 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.054842    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="480cf0fa-64a7-4107-a0e3-80224d4e94f3" containerName="authorino"
Jul 10 20:51:24.054922 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.054852    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9a298d-e83f-4e9e-a789-b8a9a8b409b3" containerName="authorino"
Jul 10 20:51:24.054922 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.054858    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9a298d-e83f-4e9e-a789-b8a9a8b409b3" containerName="authorino"
Jul 10 20:51:24.055052 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.054926    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="480cf0fa-64a7-4107-a0e3-80224d4e94f3" containerName="authorino"
Jul 10 20:51:24.055052 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.054935    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9a298d-e83f-4e9e-a789-b8a9a8b409b3" containerName="authorino"
Jul 10 20:51:24.057941 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.057924    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.060754 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.060725    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 20:51:24.061989 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.061973    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 10 20:51:24.062054 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.061985    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qhmhg\""
Jul 10 20:51:24.062054 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.061988    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 20:51:24.067816 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.067796    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"]
Jul 10 20:51:24.085893 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.085869    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.086030 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.085903    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm22l\" (UniqueName: \"kubernetes.io/projected/9cd84d43-c27e-4f09-befd-928d7c99dd45-kube-api-access-jm22l\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.086030 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.085929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd84d43-c27e-4f09-befd-928d7c99dd45-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.086030 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.086018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.086181 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.086065    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.086181 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.086110    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.187379 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.187350    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.187553 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.187390    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.187553 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.187407    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.187553 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.187462    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.187553 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.187497    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm22l\" (UniqueName: \"kubernetes.io/projected/9cd84d43-c27e-4f09-befd-928d7c99dd45-kube-api-access-jm22l\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.187553 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.187531    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd84d43-c27e-4f09-befd-928d7c99dd45-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.187877 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.187845    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.188011 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.187885    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.188011 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.187931    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.189897 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.189877    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd84d43-c27e-4f09-befd-928d7c99dd45-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.190087 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.190057    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd84d43-c27e-4f09-befd-928d7c99dd45-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.195846 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.195815    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm22l\" (UniqueName: \"kubernetes.io/projected/9cd84d43-c27e-4f09-befd-928d7c99dd45-kube-api-access-jm22l\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jtq7t\" (UID: \"9cd84d43-c27e-4f09-befd-928d7c99dd45\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.370393 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.370308    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:24.503515 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.503490    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"]
Jul 10 20:51:24.505158 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:51:24.505127    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd84d43_c27e_4f09_befd_928d7c99dd45.slice/crio-e6e44bb580db207eadecf1e8e1c8859f2de4dd35e9e5e2001ceb67c8f0a27dd3 WatchSource:0}: Error finding container e6e44bb580db207eadecf1e8e1c8859f2de4dd35e9e5e2001ceb67c8f0a27dd3: Status 404 returned error can't find the container with id e6e44bb580db207eadecf1e8e1c8859f2de4dd35e9e5e2001ceb67c8f0a27dd3
Jul 10 20:51:24.641270 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:24.641181    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t" event={"ID":"9cd84d43-c27e-4f09-befd-928d7c99dd45","Type":"ContainerStarted","Data":"e6e44bb580db207eadecf1e8e1c8859f2de4dd35e9e5e2001ceb67c8f0a27dd3"}
Jul 10 20:51:26.654015 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:26.653981    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t" event={"ID":"9cd84d43-c27e-4f09-befd-928d7c99dd45","Type":"ContainerStarted","Data":"ccb1e6bfc83f3b5b622c95038ed6cbdd28fce2264b3e2cbe6002bd1712c963fb"}
Jul 10 20:51:26.673246 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:26.673194    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t" podStartSLOduration=1.561302457 podStartE2EDuration="2.67317917s" totalImagesPullingTime="1.111876713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:51:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:51:26.671283042 +0000 UTC m=+823.164079220" watchObservedRunningTime="2026-07-10 20:51:26.67317917 +0000 UTC m=+823.165975349"
Jul 10 20:51:34.371142 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:34.371108    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:34.371647 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:34.371196    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:34.383522 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:34.383501    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:51:34.708747 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:51:34.708652    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jtq7t"
Jul 10 20:52:04.950741 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:04.950704    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"]
Jul 10 20:52:04.956858 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:04.956821    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:04.959997 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:04.959974    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 10 20:52:04.961344 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:04.961322    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"]
Jul 10 20:52:05.051746 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.051715    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxb4r\" (UniqueName: \"kubernetes.io/projected/03765a0f-c646-4c3d-81d2-abd9d4f75937-kube-api-access-dxb4r\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.051746 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.051746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.051924 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.051841    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.051924 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.051866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.051924 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.051905    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.052053 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.051939    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03765a0f-c646-4c3d-81d2-abd9d4f75937-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.153127 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.153089    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.153127 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.153126    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.153358 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.153150    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.153358 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.153175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03765a0f-c646-4c3d-81d2-abd9d4f75937-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.153358 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.153207    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxb4r\" (UniqueName: \"kubernetes.io/projected/03765a0f-c646-4c3d-81d2-abd9d4f75937-kube-api-access-dxb4r\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.153358 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.153232    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.153600 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.153535    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.153672 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.153626    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.153718 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.153675    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.155520 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.155500    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03765a0f-c646-4c3d-81d2-abd9d4f75937-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.155831 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.155814    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03765a0f-c646-4c3d-81d2-abd9d4f75937-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.161002 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.160976    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxb4r\" (UniqueName: \"kubernetes.io/projected/03765a0f-c646-4c3d-81d2-abd9d4f75937-kube-api-access-dxb4r\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x\" (UID: \"03765a0f-c646-4c3d-81d2-abd9d4f75937\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.266835 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.266759    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:05.401222 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.401192    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"]
Jul 10 20:52:05.402782 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:52:05.402760    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03765a0f_c646_4c3d_81d2_abd9d4f75937.slice/crio-37b3c0326836841c5546a257f1a8ef1bd4546899524312eeee5137970229595d WatchSource:0}: Error finding container 37b3c0326836841c5546a257f1a8ef1bd4546899524312eeee5137970229595d: Status 404 returned error can't find the container with id 37b3c0326836841c5546a257f1a8ef1bd4546899524312eeee5137970229595d
Jul 10 20:52:05.859520 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.859436    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x" event={"ID":"03765a0f-c646-4c3d-81d2-abd9d4f75937","Type":"ContainerStarted","Data":"b8e49609ac916072819a27addcc67f2c0abec5b7d08dd5e3e2b05b9c20eaef07"}
Jul 10 20:52:05.859520 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.859484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x" event={"ID":"03765a0f-c646-4c3d-81d2-abd9d4f75937","Type":"ContainerStarted","Data":"37b3c0326836841c5546a257f1a8ef1bd4546899524312eeee5137970229595d"}
Jul 10 20:52:05.877438 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:05.877391    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x" podStartSLOduration=1.70532219 podStartE2EDuration="1.877377048s" totalImagesPullingTime="172.054858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:52:05.876074108 +0000 UTC m=+862.368870286" watchObservedRunningTime="2026-07-10 20:52:05.877377048 +0000 UTC m=+862.370173228"
Jul 10 20:52:15.267440 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:15.267396    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:15.267910 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:15.267582    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:15.280138 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:15.280118    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:15.926909 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:15.926882    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x"
Jul 10 20:52:44.167137 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:44.167026    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:52:44.172197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:44.170070    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:52:44.172197 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:44.172130    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:52:44.175321 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:52:44.175302    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:54:09.541186 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.541147    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66bfb49bfc-q5j4g"]
Jul 10 20:54:09.544526 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.544508    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66bfb49bfc-q5j4g"
Jul 10 20:54:09.559120 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.559092    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66bfb49bfc-q5j4g"]
Jul 10 20:54:09.651301 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.651262    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78c17974-b45b-442a-af5a-b9f9bba6773a-tls-cert\") pod \"authorino-66bfb49bfc-q5j4g\" (UID: \"78c17974-b45b-442a-af5a-b9f9bba6773a\") " pod="kuadrant-system/authorino-66bfb49bfc-q5j4g"
Jul 10 20:54:09.651506 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.651312    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vskcg\" (UniqueName: \"kubernetes.io/projected/78c17974-b45b-442a-af5a-b9f9bba6773a-kube-api-access-vskcg\") pod \"authorino-66bfb49bfc-q5j4g\" (UID: \"78c17974-b45b-442a-af5a-b9f9bba6773a\") " pod="kuadrant-system/authorino-66bfb49bfc-q5j4g"
Jul 10 20:54:09.752738 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.752691    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78c17974-b45b-442a-af5a-b9f9bba6773a-tls-cert\") pod \"authorino-66bfb49bfc-q5j4g\" (UID: \"78c17974-b45b-442a-af5a-b9f9bba6773a\") " pod="kuadrant-system/authorino-66bfb49bfc-q5j4g"
Jul 10 20:54:09.752933 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.752766    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vskcg\" (UniqueName: \"kubernetes.io/projected/78c17974-b45b-442a-af5a-b9f9bba6773a-kube-api-access-vskcg\") pod \"authorino-66bfb49bfc-q5j4g\" (UID: \"78c17974-b45b-442a-af5a-b9f9bba6773a\") " pod="kuadrant-system/authorino-66bfb49bfc-q5j4g"
Jul 10 20:54:09.755357 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.755332    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78c17974-b45b-442a-af5a-b9f9bba6773a-tls-cert\") pod \"authorino-66bfb49bfc-q5j4g\" (UID: \"78c17974-b45b-442a-af5a-b9f9bba6773a\") " pod="kuadrant-system/authorino-66bfb49bfc-q5j4g"
Jul 10 20:54:09.760429 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.760401    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vskcg\" (UniqueName: \"kubernetes.io/projected/78c17974-b45b-442a-af5a-b9f9bba6773a-kube-api-access-vskcg\") pod \"authorino-66bfb49bfc-q5j4g\" (UID: \"78c17974-b45b-442a-af5a-b9f9bba6773a\") " pod="kuadrant-system/authorino-66bfb49bfc-q5j4g"
Jul 10 20:54:09.853347 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.853303    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66bfb49bfc-q5j4g"
Jul 10 20:54:09.992172 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.992148    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66bfb49bfc-q5j4g"]
Jul 10 20:54:09.994132 ip-10-0-141-228 kubenswrapper[2565]: W0710 20:54:09.994100    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78c17974_b45b_442a_af5a_b9f9bba6773a.slice/crio-038341580e6afe7d43343baca3239246f9bde30877cfb0fb45e1b9e9b58a12a6 WatchSource:0}: Error finding container 038341580e6afe7d43343baca3239246f9bde30877cfb0fb45e1b9e9b58a12a6: Status 404 returned error can't find the container with id 038341580e6afe7d43343baca3239246f9bde30877cfb0fb45e1b9e9b58a12a6
Jul 10 20:54:09.995442 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:09.995426    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 20:54:10.531617 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.531577    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66bfb49bfc-q5j4g" event={"ID":"78c17974-b45b-442a-af5a-b9f9bba6773a","Type":"ContainerStarted","Data":"3c31514e222a3cbc4821b1958839a61b61ba186164e08d8c4c6e5f774783f8f1"}
Jul 10 20:54:10.531875 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.531815    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66bfb49bfc-q5j4g" event={"ID":"78c17974-b45b-442a-af5a-b9f9bba6773a","Type":"ContainerStarted","Data":"038341580e6afe7d43343baca3239246f9bde30877cfb0fb45e1b9e9b58a12a6"}
Jul 10 20:54:10.548025 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.547957    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66bfb49bfc-q5j4g" podStartSLOduration=1.156046261 podStartE2EDuration="1.547931891s" totalImagesPullingTime="391.88563ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:54:10.546078937 +0000 UTC m=+987.038875114" watchObservedRunningTime="2026-07-10 20:54:10.547931891 +0000 UTC m=+987.040728071"
Jul 10 20:54:10.577366 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.577329    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8c5976d96-lqprb"]
Jul 10 20:54:10.577605 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.577574    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-8c5976d96-lqprb" podUID="ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb" containerName="authorino" containerID="cri-o://8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d" gracePeriod=30
Jul 10 20:54:10.843815 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.843781    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:54:10.964280 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.964174    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c69tt\" (UniqueName: \"kubernetes.io/projected/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-kube-api-access-c69tt\") pod \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\" (UID: \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\") "
Jul 10 20:54:10.964280 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.964252    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-tls-cert\") pod \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\" (UID: \"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb\") "
Jul 10 20:54:10.966476 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.966443    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-kube-api-access-c69tt" (OuterVolumeSpecName: "kube-api-access-c69tt") pod "ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb" (UID: "ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb"). InnerVolumeSpecName "kube-api-access-c69tt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:54:10.976723 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:10.976682    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb" (UID: "ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:54:11.065346 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.065304    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c69tt\" (UniqueName: \"kubernetes.io/projected/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-kube-api-access-c69tt\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:54:11.065346 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.065335    2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb-tls-cert\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 20:54:11.538693 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.538653    2565 generic.go:356] "Generic (PLEG): container finished" podID="ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb" containerID="8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d" exitCode=0
Jul 10 20:54:11.538867 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.538738    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c5976d96-lqprb"
Jul 10 20:54:11.538867 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.538758    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c5976d96-lqprb" event={"ID":"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb","Type":"ContainerDied","Data":"8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d"}
Jul 10 20:54:11.538867 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.538825    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c5976d96-lqprb" event={"ID":"ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb","Type":"ContainerDied","Data":"3537133b937c15f27d3e7cce72e2ceeb64f90be8b99c852252d9d2cf52e79852"}
Jul 10 20:54:11.538867 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.538849    2565 scope.go:117] "RemoveContainer" containerID="8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d"
Jul 10 20:54:11.549189 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.549165    2565 scope.go:117] "RemoveContainer" containerID="8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d"
Jul 10 20:54:11.549529 ip-10-0-141-228 kubenswrapper[2565]: E0710 20:54:11.549507    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d\": container with ID starting with 8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d not found: ID does not exist" containerID="8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d"
Jul 10 20:54:11.549616 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.549539    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d"} err="failed to get container status \"8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d\": rpc error: code = NotFound desc = could not find container \"8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d\": container with ID starting with 8234901d7db3c24d195b9523d5e5b3208f6182868b287469fe53d42ddbbbe56d not found: ID does not exist"
Jul 10 20:54:11.561599 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.561550    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8c5976d96-lqprb"]
Jul 10 20:54:11.566797 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.566774    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8c5976d96-lqprb"]
Jul 10 20:54:11.961288 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:54:11.961242    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb" path="/var/lib/kubelet/pods/ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb/volumes"
Jul 10 20:57:44.208054 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:57:44.207942    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:57:44.212195 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:57:44.210821    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 20:57:44.213426 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:57:44.213407    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 20:57:44.216338 ip-10-0-141-228 kubenswrapper[2565]: I0710 20:57:44.216318    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:02:44.247813 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:02:44.247695    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:02:44.252498 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:02:44.252473    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:02:44.254175 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:02:44.254148    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:02:44.258792 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:02:44.258773    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:07:44.289027 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:07:44.288923    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:07:44.296074 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:07:44.293756    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:07:44.296074 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:07:44.294538    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:07:44.299177 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:07:44.299159    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:09:48.947524 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:48.947489    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"]
Jul 10 21:09:48.947991 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:48.947880    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb" containerName="authorino"
Jul 10 21:09:48.947991 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:48.947891    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb" containerName="authorino"
Jul 10 21:09:48.947991 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:48.947953    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad6747e3-acdc-4ae6-b6bf-37d2ffb5b6eb" containerName="authorino"
Jul 10 21:09:48.950921 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:48.950903    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"
Jul 10 21:09:48.953669 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:48.953647    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:09:48.953796 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:48.953694    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 21:09:48.958290 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:48.958265    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"]
Jul 10 21:09:48.982013 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:48.981986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea09632b-40c6-4f53-871b-2d5d7b6b6b07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6fde404a-822k4\" (UID: \"ea09632b-40c6-4f53-871b-2d5d7b6b6b07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"
Jul 10 21:09:49.082523 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:49.082486    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea09632b-40c6-4f53-871b-2d5d7b6b6b07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6fde404a-822k4\" (UID: \"ea09632b-40c6-4f53-871b-2d5d7b6b6b07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"
Jul 10 21:09:49.083170 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:49.083148    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea09632b-40c6-4f53-871b-2d5d7b6b6b07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6fde404a-822k4\" (UID: \"ea09632b-40c6-4f53-871b-2d5d7b6b6b07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"
Jul 10 21:09:49.262508 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:49.262425    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"
Jul 10 21:09:49.398851 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:49.398825    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"]
Jul 10 21:09:49.401215 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:09:49.401184    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea09632b_40c6_4f53_871b_2d5d7b6b6b07.slice/crio-1e36cbea3070d2f343d85a2b4a9363e74403cfe47c8e566a315a1aa5c3cadb3f WatchSource:0}: Error finding container 1e36cbea3070d2f343d85a2b4a9363e74403cfe47c8e566a315a1aa5c3cadb3f: Status 404 returned error can't find the container with id 1e36cbea3070d2f343d85a2b4a9363e74403cfe47c8e566a315a1aa5c3cadb3f
Jul 10 21:09:49.403225 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:49.403206    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 21:09:49.679381 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:49.679343    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4" event={"ID":"ea09632b-40c6-4f53-871b-2d5d7b6b6b07","Type":"ContainerStarted","Data":"1e36cbea3070d2f343d85a2b4a9363e74403cfe47c8e566a315a1aa5c3cadb3f"}
Jul 10 21:09:51.692732 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:51.692690    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4" event={"ID":"ea09632b-40c6-4f53-871b-2d5d7b6b6b07","Type":"ContainerStarted","Data":"bcb11e5ccef3c6ce1f5b142c5547fe1866299fe634a43c2b65559c1645940069"}
Jul 10 21:09:51.713122 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:51.713076    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4" podStartSLOduration=1.757801838 podStartE2EDuration="3.713061919s" totalImagesPullingTime="1.955260081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:09:51.710056835 +0000 UTC m=+1928.202853014" watchObservedRunningTime="2026-07-10 21:09:51.713061919 +0000 UTC m=+1928.205858142"
Jul 10 21:09:52.699282 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:52.699247    2565 generic.go:356] "Generic (PLEG): container finished" podID="ea09632b-40c6-4f53-871b-2d5d7b6b6b07" containerID="bcb11e5ccef3c6ce1f5b142c5547fe1866299fe634a43c2b65559c1645940069" exitCode=0
Jul 10 21:09:52.699692 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:52.699330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4" event={"ID":"ea09632b-40c6-4f53-871b-2d5d7b6b6b07","Type":"ContainerDied","Data":"bcb11e5ccef3c6ce1f5b142c5547fe1866299fe634a43c2b65559c1645940069"}
Jul 10 21:09:53.834303 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:53.834279    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"
Jul 10 21:09:53.922404 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:53.922377    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea09632b-40c6-4f53-871b-2d5d7b6b6b07-maas-api-service-ca\") pod \"ea09632b-40c6-4f53-871b-2d5d7b6b6b07\" (UID: \"ea09632b-40c6-4f53-871b-2d5d7b6b6b07\") "
Jul 10 21:09:53.922732 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:53.922712    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea09632b-40c6-4f53-871b-2d5d7b6b6b07-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ea09632b-40c6-4f53-871b-2d5d7b6b6b07" (UID: "ea09632b-40c6-4f53-871b-2d5d7b6b6b07"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 21:09:54.023581 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:54.023533    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea09632b-40c6-4f53-871b-2d5d7b6b6b07-maas-api-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:09:54.710618 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:54.710590    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4"
Jul 10 21:09:54.710816 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:54.710586    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-822k4" event={"ID":"ea09632b-40c6-4f53-871b-2d5d7b6b6b07","Type":"ContainerDied","Data":"1e36cbea3070d2f343d85a2b4a9363e74403cfe47c8e566a315a1aa5c3cadb3f"}
Jul 10 21:09:54.710816 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:09:54.710747    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e36cbea3070d2f343d85a2b4a9363e74403cfe47c8e566a315a1aa5c3cadb3f"
Jul 10 21:10:11.276633 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.276597    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"]
Jul 10 21:10:11.277033 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.277022    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea09632b-40c6-4f53-871b-2d5d7b6b6b07" containerName="revoke-keys"
Jul 10 21:10:11.277075 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.277034    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea09632b-40c6-4f53-871b-2d5d7b6b6b07" containerName="revoke-keys"
Jul 10 21:10:11.277109 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.277089    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea09632b-40c6-4f53-871b-2d5d7b6b6b07" containerName="revoke-keys"
Jul 10 21:10:11.285249 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.285214    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"
Jul 10 21:10:11.287011 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.286967    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"]
Jul 10 21:10:11.288078 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.288053    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 21:10:11.288203 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.288117    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:10:11.373179 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.373148    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c3663f-65f3-4a60-a224-ea3ef54b29ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s\" (UID: \"a9c3663f-65f3-4a60-a224-ea3ef54b29ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"
Jul 10 21:10:11.473896 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.473865    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c3663f-65f3-4a60-a224-ea3ef54b29ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s\" (UID: \"a9c3663f-65f3-4a60-a224-ea3ef54b29ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"
Jul 10 21:10:11.474527 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.474506    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c3663f-65f3-4a60-a224-ea3ef54b29ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s\" (UID: \"a9c3663f-65f3-4a60-a224-ea3ef54b29ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"
Jul 10 21:10:11.597977 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.597941    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"
Jul 10 21:10:11.731004 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.730973    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"]
Jul 10 21:10:11.732494 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:10:11.732468    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c3663f_65f3_4a60_a224_ea3ef54b29ac.slice/crio-6bba902eab4af92e8615352d8a3be2df88bbe199f6ced9ad0ecb63177b557458 WatchSource:0}: Error finding container 6bba902eab4af92e8615352d8a3be2df88bbe199f6ced9ad0ecb63177b557458: Status 404 returned error can't find the container with id 6bba902eab4af92e8615352d8a3be2df88bbe199f6ced9ad0ecb63177b557458
Jul 10 21:10:11.800483 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:11.800451    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s" event={"ID":"a9c3663f-65f3-4a60-a224-ea3ef54b29ac","Type":"ContainerStarted","Data":"6bba902eab4af92e8615352d8a3be2df88bbe199f6ced9ad0ecb63177b557458"}
Jul 10 21:10:12.807273 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:12.807238    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerID="572a5b4888abc0673b745bb8d2638d2b33c83c0bdca2811279338cf3362dcdda" exitCode=7
Jul 10 21:10:12.807679 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:12.807326    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s" event={"ID":"a9c3663f-65f3-4a60-a224-ea3ef54b29ac","Type":"ContainerDied","Data":"572a5b4888abc0673b745bb8d2638d2b33c83c0bdca2811279338cf3362dcdda"}
Jul 10 21:10:12.807679 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:12.807539    2565 scope.go:117] "RemoveContainer" containerID="572a5b4888abc0673b745bb8d2638d2b33c83c0bdca2811279338cf3362dcdda"
Jul 10 21:10:13.815106 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:13.815071    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerID="5cc8bd70bf7aa520009bca34a40cb0942bb13ec2bb6833fa77022abdb869334c" exitCode=7
Jul 10 21:10:13.815509 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:13.815148    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s" event={"ID":"a9c3663f-65f3-4a60-a224-ea3ef54b29ac","Type":"ContainerDied","Data":"5cc8bd70bf7aa520009bca34a40cb0942bb13ec2bb6833fa77022abdb869334c"}
Jul 10 21:10:13.815509 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:13.815202    2565 scope.go:117] "RemoveContainer" containerID="572a5b4888abc0673b745bb8d2638d2b33c83c0bdca2811279338cf3362dcdda"
Jul 10 21:10:13.815509 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:13.815402    2565 scope.go:117] "RemoveContainer" containerID="5cc8bd70bf7aa520009bca34a40cb0942bb13ec2bb6833fa77022abdb869334c"
Jul 10 21:10:13.815674 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:10:13.815618    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s_odh-ai-gateway-infra(a9c3663f-65f3-4a60-a224-ea3ef54b29ac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac"
Jul 10 21:10:14.822602 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:14.822546    2565 scope.go:117] "RemoveContainer" containerID="5cc8bd70bf7aa520009bca34a40cb0942bb13ec2bb6833fa77022abdb869334c"
Jul 10 21:10:14.823001 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:10:14.822780    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s_odh-ai-gateway-infra(a9c3663f-65f3-4a60-a224-ea3ef54b29ac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac"
Jul 10 21:10:29.951204 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:29.951172    2565 scope.go:117] "RemoveContainer" containerID="5cc8bd70bf7aa520009bca34a40cb0942bb13ec2bb6833fa77022abdb869334c"
Jul 10 21:10:30.907432 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:30.907347    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerID="8d0f173dd7c2e3d40373f8fd3a0b14ae384b562c8764105dc5701246e989f185" exitCode=0
Jul 10 21:10:30.907604 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:30.907423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s" event={"ID":"a9c3663f-65f3-4a60-a224-ea3ef54b29ac","Type":"ContainerDied","Data":"8d0f173dd7c2e3d40373f8fd3a0b14ae384b562c8764105dc5701246e989f185"}
Jul 10 21:10:30.907604 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:30.907484    2565 scope.go:117] "RemoveContainer" containerID="5cc8bd70bf7aa520009bca34a40cb0942bb13ec2bb6833fa77022abdb869334c"
Jul 10 21:10:31.939700 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:31.939670    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"]
Jul 10 21:10:32.046790 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:32.046765    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"
Jul 10 21:10:32.168364 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:32.168266    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c3663f-65f3-4a60-a224-ea3ef54b29ac-maas-api-service-ca\") pod \"a9c3663f-65f3-4a60-a224-ea3ef54b29ac\" (UID: \"a9c3663f-65f3-4a60-a224-ea3ef54b29ac\") "
Jul 10 21:10:32.168699 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:32.168675    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9c3663f-65f3-4a60-a224-ea3ef54b29ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a9c3663f-65f3-4a60-a224-ea3ef54b29ac" (UID: "a9c3663f-65f3-4a60-a224-ea3ef54b29ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 21:10:32.269214 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:32.269179    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c3663f-65f3-4a60-a224-ea3ef54b29ac-maas-api-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:10:32.927428 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:32.927395    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"
Jul 10 21:10:32.927628 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:32.927429    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s" event={"ID":"a9c3663f-65f3-4a60-a224-ea3ef54b29ac","Type":"ContainerDied","Data":"6bba902eab4af92e8615352d8a3be2df88bbe199f6ced9ad0ecb63177b557458"}
Jul 10 21:10:32.927628 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:32.927509    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bba902eab4af92e8615352d8a3be2df88bbe199f6ced9ad0ecb63177b557458"
Jul 10 21:10:32.955030 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:32.955004    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"]
Jul 10 21:10:32.957077 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:32.957056    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-mcj7s"]
Jul 10 21:10:33.956625 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:10:33.956593    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" path="/var/lib/kubelet/pods/a9c3663f-65f3-4a60-a224-ea3ef54b29ac/volumes"
Jul 10 21:11:43.245875 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.245834    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"]
Jul 10 21:11:43.246419 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.246385    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerName="revoke-keys"
Jul 10 21:11:43.246419 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.246406    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerName="revoke-keys"
Jul 10 21:11:43.246419 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.246418    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerName="revoke-keys"
Jul 10 21:11:43.246625 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.246427    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerName="revoke-keys"
Jul 10 21:11:43.246625 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.246449    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerName="revoke-keys"
Jul 10 21:11:43.246625 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.246458    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerName="revoke-keys"
Jul 10 21:11:43.246625 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.246588    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerName="revoke-keys"
Jul 10 21:11:43.246625 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.246602    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerName="revoke-keys"
Jul 10 21:11:43.249684 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.249664    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"
Jul 10 21:11:43.253164 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.253015    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 21:11:43.253164 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.253028    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:11:43.254657 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.254634    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"]
Jul 10 21:11:43.310351 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.310318    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a80be958-f8fb-4f08-9e66-98e297e18268-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-183404a5-k58bl\" (UID: \"a80be958-f8fb-4f08-9e66-98e297e18268\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"
Jul 10 21:11:43.411339 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.411307    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a80be958-f8fb-4f08-9e66-98e297e18268-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-183404a5-k58bl\" (UID: \"a80be958-f8fb-4f08-9e66-98e297e18268\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"
Jul 10 21:11:43.411933 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.411916    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a80be958-f8fb-4f08-9e66-98e297e18268-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-183404a5-k58bl\" (UID: \"a80be958-f8fb-4f08-9e66-98e297e18268\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"
Jul 10 21:11:43.561071 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.561040    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"
Jul 10 21:11:43.694523 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:43.694494    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"]
Jul 10 21:11:43.696313 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:11:43.696289    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80be958_f8fb_4f08_9e66_98e297e18268.slice/crio-a65bd39d272578d16613bc98acc39279b822853c0279583cc8443e31d34194f7 WatchSource:0}: Error finding container a65bd39d272578d16613bc98acc39279b822853c0279583cc8443e31d34194f7: Status 404 returned error can't find the container with id a65bd39d272578d16613bc98acc39279b822853c0279583cc8443e31d34194f7
Jul 10 21:11:44.331787 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:44.331753    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" event={"ID":"a80be958-f8fb-4f08-9e66-98e297e18268","Type":"ContainerStarted","Data":"83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6"}
Jul 10 21:11:44.332091 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:44.331820    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" event={"ID":"a80be958-f8fb-4f08-9e66-98e297e18268","Type":"ContainerStarted","Data":"a65bd39d272578d16613bc98acc39279b822853c0279583cc8443e31d34194f7"}
Jul 10 21:11:44.332091 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:44.331908    2565 scope.go:117] "RemoveContainer" containerID="83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6"
Jul 10 21:11:45.338995 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:45.338961    2565 generic.go:356] "Generic (PLEG): container finished" podID="a80be958-f8fb-4f08-9e66-98e297e18268" containerID="83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6" exitCode=7
Jul 10 21:11:45.338995 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:45.338986    2565 generic.go:356] "Generic (PLEG): container finished" podID="a80be958-f8fb-4f08-9e66-98e297e18268" containerID="b6bfd2ab7fe0d672f3ab654256830229a4a8872f984aafb1d2c52de53e1a2656" exitCode=7
Jul 10 21:11:45.339483 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:45.339016    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" event={"ID":"a80be958-f8fb-4f08-9e66-98e297e18268","Type":"ContainerDied","Data":"83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6"}
Jul 10 21:11:45.339483 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:45.339059    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" event={"ID":"a80be958-f8fb-4f08-9e66-98e297e18268","Type":"ContainerDied","Data":"b6bfd2ab7fe0d672f3ab654256830229a4a8872f984aafb1d2c52de53e1a2656"}
Jul 10 21:11:45.339483 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:45.339080    2565 scope.go:117] "RemoveContainer" containerID="83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6"
Jul 10 21:11:45.339483 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:45.339251    2565 scope.go:117] "RemoveContainer" containerID="b6bfd2ab7fe0d672f3ab654256830229a4a8872f984aafb1d2c52de53e1a2656"
Jul 10 21:11:45.339483 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:11:45.339454    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-183404a5-k58bl_odh-ai-gateway-infra(a80be958-f8fb-4f08-9e66-98e297e18268)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" podUID="a80be958-f8fb-4f08-9e66-98e297e18268"
Jul 10 21:11:45.350485 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:45.350465    2565 scope.go:117] "RemoveContainer" containerID="83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6"
Jul 10 21:11:45.350764 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:11:45.350747    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6\": container with ID starting with 83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6 not found: ID does not exist" containerID="83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6"
Jul 10 21:11:45.350834 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:45.350776    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6"} err="failed to get container status \"83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6\": rpc error: code = NotFound desc = could not find container \"83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6\": container with ID starting with 83d4448d7b6bfff9067d00f9950185110e57b6cde12bf18e1d27b03e33fe55f6 not found: ID does not exist"
Jul 10 21:11:46.346056 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:46.346032    2565 scope.go:117] "RemoveContainer" containerID="b6bfd2ab7fe0d672f3ab654256830229a4a8872f984aafb1d2c52de53e1a2656"
Jul 10 21:11:46.346433 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:11:46.346208    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-183404a5-k58bl_odh-ai-gateway-infra(a80be958-f8fb-4f08-9e66-98e297e18268)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" podUID="a80be958-f8fb-4f08-9e66-98e297e18268"
Jul 10 21:11:58.951980 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:58.951947    2565 scope.go:117] "RemoveContainer" containerID="b6bfd2ab7fe0d672f3ab654256830229a4a8872f984aafb1d2c52de53e1a2656"
Jul 10 21:11:59.417756 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:59.417724    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" event={"ID":"a80be958-f8fb-4f08-9e66-98e297e18268","Type":"ContainerStarted","Data":"b53905982c7e01e1ae8bead796904ce4533fdd98d90c61b456802e67ab4cb355"}
Jul 10 21:11:59.435629 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:11:59.435573    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" podStartSLOduration=16.435542935 podStartE2EDuration="16.435542935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:11:59.434144369 +0000 UTC m=+2055.926940548" watchObservedRunningTime="2026-07-10 21:11:59.435542935 +0000 UTC m=+2055.928339113"
Jul 10 21:12:00.424609 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:00.424553    2565 generic.go:356] "Generic (PLEG): container finished" podID="a80be958-f8fb-4f08-9e66-98e297e18268" containerID="b53905982c7e01e1ae8bead796904ce4533fdd98d90c61b456802e67ab4cb355" exitCode=0
Jul 10 21:12:00.425068 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:00.424626    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" event={"ID":"a80be958-f8fb-4f08-9e66-98e297e18268","Type":"ContainerDied","Data":"b53905982c7e01e1ae8bead796904ce4533fdd98d90c61b456802e67ab4cb355"}
Jul 10 21:12:00.425068 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:00.424716    2565 scope.go:117] "RemoveContainer" containerID="b6bfd2ab7fe0d672f3ab654256830229a4a8872f984aafb1d2c52de53e1a2656"
Jul 10 21:12:00.449991 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:00.449964    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"]
Jul 10 21:12:01.564942 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:01.564915    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"
Jul 10 21:12:01.573171 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:01.573148    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a80be958-f8fb-4f08-9e66-98e297e18268-maas-api-service-ca\") pod \"a80be958-f8fb-4f08-9e66-98e297e18268\" (UID: \"a80be958-f8fb-4f08-9e66-98e297e18268\") "
Jul 10 21:12:01.573464 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:01.573442    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80be958-f8fb-4f08-9e66-98e297e18268-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a80be958-f8fb-4f08-9e66-98e297e18268" (UID: "a80be958-f8fb-4f08-9e66-98e297e18268"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 21:12:01.673950 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:01.673917    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a80be958-f8fb-4f08-9e66-98e297e18268-maas-api-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:12:02.438103 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:02.438074    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"
Jul 10 21:12:02.438278 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:02.438119    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl" event={"ID":"a80be958-f8fb-4f08-9e66-98e297e18268","Type":"ContainerDied","Data":"a65bd39d272578d16613bc98acc39279b822853c0279583cc8443e31d34194f7"}
Jul 10 21:12:02.438278 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:02.438168    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a65bd39d272578d16613bc98acc39279b822853c0279583cc8443e31d34194f7"
Jul 10 21:12:02.476497 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:02.476465    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"]
Jul 10 21:12:02.478504 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:02.478483    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-k58bl"]
Jul 10 21:12:03.957593 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:03.957538    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" path="/var/lib/kubelet/pods/a80be958-f8fb-4f08-9e66-98e297e18268/volumes"
Jul 10 21:12:44.334094 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:44.334062    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:12:44.339603 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:44.339581    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:12:44.339881 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:44.339865    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:12:44.344987 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:12:44.344969    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:14:51.018484 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.018447    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"]
Jul 10 21:14:51.018968 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.018820    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" containerName="revoke-keys"
Jul 10 21:14:51.018968 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.018831    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" containerName="revoke-keys"
Jul 10 21:14:51.018968 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.018858    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" containerName="revoke-keys"
Jul 10 21:14:51.018968 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.018864    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" containerName="revoke-keys"
Jul 10 21:14:51.018968 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.018920    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" containerName="revoke-keys"
Jul 10 21:14:51.018968 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.018930    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" containerName="revoke-keys"
Jul 10 21:14:51.018968 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.018937    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" containerName="revoke-keys"
Jul 10 21:14:51.018968 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.018945    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9c3663f-65f3-4a60-a224-ea3ef54b29ac" containerName="revoke-keys"
Jul 10 21:14:51.021907 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.021889    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"
Jul 10 21:14:51.025907 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.025882    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 21:14:51.026650 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.026631    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:14:51.033430 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.033405    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"]
Jul 10 21:14:51.184199 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.184165    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6454bc2-2276-4d0b-acad-462ef4284c6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dd381648-psl2w\" (UID: \"d6454bc2-2276-4d0b-acad-462ef4284c6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"
Jul 10 21:14:51.285626 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.285530    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6454bc2-2276-4d0b-acad-462ef4284c6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dd381648-psl2w\" (UID: \"d6454bc2-2276-4d0b-acad-462ef4284c6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"
Jul 10 21:14:51.286177 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.286155    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6454bc2-2276-4d0b-acad-462ef4284c6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dd381648-psl2w\" (UID: \"d6454bc2-2276-4d0b-acad-462ef4284c6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"
Jul 10 21:14:51.332857 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.332826    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"
Jul 10 21:14:51.478740 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:51.478713    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"]
Jul 10 21:14:51.480733 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:14:51.480701    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6454bc2_2276_4d0b_acad_462ef4284c6f.slice/crio-8ccdcba4767ca1df8f372f332be7c0769c9fa207ad11e1540ea286287f4a75d8 WatchSource:0}: Error finding container 8ccdcba4767ca1df8f372f332be7c0769c9fa207ad11e1540ea286287f4a75d8: Status 404 returned error can't find the container with id 8ccdcba4767ca1df8f372f332be7c0769c9fa207ad11e1540ea286287f4a75d8
Jul 10 21:14:52.397693 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:52.397651    2565 generic.go:356] "Generic (PLEG): container finished" podID="d6454bc2-2276-4d0b-acad-462ef4284c6f" containerID="bd9e687f53543db7086c3ea237514deebb2212125d6ff3e455dd4665a2201b6b" exitCode=0
Jul 10 21:14:52.398059 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:52.397733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w" event={"ID":"d6454bc2-2276-4d0b-acad-462ef4284c6f","Type":"ContainerDied","Data":"bd9e687f53543db7086c3ea237514deebb2212125d6ff3e455dd4665a2201b6b"}
Jul 10 21:14:52.398059 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:52.397789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w" event={"ID":"d6454bc2-2276-4d0b-acad-462ef4284c6f","Type":"ContainerStarted","Data":"8ccdcba4767ca1df8f372f332be7c0769c9fa207ad11e1540ea286287f4a75d8"}
Jul 10 21:14:53.534066 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:53.534035    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"
Jul 10 21:14:53.710727 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:53.710624    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6454bc2-2276-4d0b-acad-462ef4284c6f-maas-api-service-ca\") pod \"d6454bc2-2276-4d0b-acad-462ef4284c6f\" (UID: \"d6454bc2-2276-4d0b-acad-462ef4284c6f\") "
Jul 10 21:14:53.710998 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:53.710975    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6454bc2-2276-4d0b-acad-462ef4284c6f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d6454bc2-2276-4d0b-acad-462ef4284c6f" (UID: "d6454bc2-2276-4d0b-acad-462ef4284c6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 21:14:53.812266 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:53.812217    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6454bc2-2276-4d0b-acad-462ef4284c6f-maas-api-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:14:54.409511 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:54.409477    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w"
Jul 10 21:14:54.409757 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:54.409509    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dd381648-psl2w" event={"ID":"d6454bc2-2276-4d0b-acad-462ef4284c6f","Type":"ContainerDied","Data":"8ccdcba4767ca1df8f372f332be7c0769c9fa207ad11e1540ea286287f4a75d8"}
Jul 10 21:14:54.409757 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:14:54.409596    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ccdcba4767ca1df8f372f332be7c0769c9fa207ad11e1540ea286287f4a75d8"
Jul 10 21:15:00.141998 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.141960    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn"]
Jul 10 21:15:00.142460 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.142325    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" containerName="revoke-keys"
Jul 10 21:15:00.142460 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.142336    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80be958-f8fb-4f08-9e66-98e297e18268" containerName="revoke-keys"
Jul 10 21:15:00.142460 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.142355    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6454bc2-2276-4d0b-acad-462ef4284c6f" containerName="revoke-keys"
Jul 10 21:15:00.142460 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.142360    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6454bc2-2276-4d0b-acad-462ef4284c6f" containerName="revoke-keys"
Jul 10 21:15:00.142460 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.142431    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6454bc2-2276-4d0b-acad-462ef4284c6f" containerName="revoke-keys"
Jul 10 21:15:00.145643 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.145625    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn"
Jul 10 21:15:00.148281 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.148248    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:15:00.155367 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.155339    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn"]
Jul 10 21:15:00.157113 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.157093    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn"
Jul 10 21:15:00.302879 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.302853    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn"]
Jul 10 21:15:00.304874 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:15:00.304844    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223c86e1_dc55_4c2d_93f2_307078dd0322.slice/crio-36370c0f7cd02d0f03cc1c996d72c6753cb8325749c52c4faec6b31f36fbc12e WatchSource:0}: Error finding container 36370c0f7cd02d0f03cc1c996d72c6753cb8325749c52c4faec6b31f36fbc12e: Status 404 returned error can't find the container with id 36370c0f7cd02d0f03cc1c996d72c6753cb8325749c52c4faec6b31f36fbc12e
Jul 10 21:15:00.443598 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:00.443472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn" event={"ID":"223c86e1-dc55-4c2d-93f2-307078dd0322","Type":"ContainerStarted","Data":"36370c0f7cd02d0f03cc1c996d72c6753cb8325749c52c4faec6b31f36fbc12e"}
Jul 10 21:15:01.451156 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:01.451121    2565 generic.go:356] "Generic (PLEG): container finished" podID="223c86e1-dc55-4c2d-93f2-307078dd0322" containerID="a7e2ced8d651bcd177f581c9026745fb8323bfa03476b210d56908b30b04ae48" exitCode=0
Jul 10 21:15:01.451529 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:01.451171    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn" event={"ID":"223c86e1-dc55-4c2d-93f2-307078dd0322","Type":"ContainerDied","Data":"a7e2ced8d651bcd177f581c9026745fb8323bfa03476b210d56908b30b04ae48"}
Jul 10 21:15:02.591026 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:02.590989    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn"
Jul 10 21:15:03.463972 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:03.463934    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn"
Jul 10 21:15:03.464157 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:03.463977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728635-gtkmn" event={"ID":"223c86e1-dc55-4c2d-93f2-307078dd0322","Type":"ContainerDied","Data":"36370c0f7cd02d0f03cc1c996d72c6753cb8325749c52c4faec6b31f36fbc12e"}
Jul 10 21:15:03.464157 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:03.464031    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36370c0f7cd02d0f03cc1c996d72c6753cb8325749c52c4faec6b31f36fbc12e"
Jul 10 21:15:49.570500 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.570464    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"]
Jul 10 21:15:49.570989 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.570871    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223c86e1-dc55-4c2d-93f2-307078dd0322" containerName="cleanup"
Jul 10 21:15:49.570989 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.570883    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="223c86e1-dc55-4c2d-93f2-307078dd0322" containerName="cleanup"
Jul 10 21:15:49.570989 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.570944    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="223c86e1-dc55-4c2d-93f2-307078dd0322" containerName="cleanup"
Jul 10 21:15:49.574031 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.574012    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.579104 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.579078    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-46c4a127\"/\"default-dockercfg-2kpjz\""
Jul 10 21:15:49.579284 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.579077    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-46c4a127\"/\"e2e-lifecycle-model-46c4a127-kserve-self-signed-certs\""
Jul 10 21:15:49.579284 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.579078    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-46c4a127\"/\"kube-root-ca.crt\""
Jul 10 21:15:49.579284 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.579089    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-46c4a127\"/\"openshift-service-ca.crt\""
Jul 10 21:15:49.586700 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.586669    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"]
Jul 10 21:15:49.722312 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.722276    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tls-certs\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.722546 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.722322    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkqnw\" (UniqueName: \"kubernetes.io/projected/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-kube-api-access-vkqnw\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.722546 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.722350    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tmp-dir\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.722546 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.722474    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-home\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.722546 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.722513    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-model-cache\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.722779 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.722612    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-dshm\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.823334 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.823232    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tls-certs\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.823334 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.823278    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkqnw\" (UniqueName: \"kubernetes.io/projected/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-kube-api-access-vkqnw\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.823334 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.823304    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tmp-dir\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.823666 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.823345    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-home\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.823666 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.823361    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-model-cache\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.823666 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.823386    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-dshm\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.823863 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.823834    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-model-cache\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.823929 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.823860    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-home\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.823929 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.823915    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tmp-dir\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.825769 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.825747    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-dshm\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.826102 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.826085    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tls-certs\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.835946 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.835920    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkqnw\" (UniqueName: \"kubernetes.io/projected/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-kube-api-access-vkqnw\") pod \"e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") " pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:49.883755 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:49.883715    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:50.042147 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:50.042114    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"]
Jul 10 21:15:50.043764 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:15:50.043733    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53b24fd4_7c2e_4ec6_aa02_c0e5fc2199f0.slice/crio-9a07082fd5c785445f17bb343e7fb90d67c0465bb9746b24125e92f6592f46f6 WatchSource:0}: Error finding container 9a07082fd5c785445f17bb343e7fb90d67c0465bb9746b24125e92f6592f46f6: Status 404 returned error can't find the container with id 9a07082fd5c785445f17bb343e7fb90d67c0465bb9746b24125e92f6592f46f6
Jul 10 21:15:50.737197 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:50.737157    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984" event={"ID":"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0","Type":"ContainerStarted","Data":"cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd"}
Jul 10 21:15:50.737197 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:50.737204    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984" event={"ID":"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0","Type":"ContainerStarted","Data":"9a07082fd5c785445f17bb343e7fb90d67c0465bb9746b24125e92f6592f46f6"}
Jul 10 21:15:59.884210 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:59.884170    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:59.884724 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:59.884227    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:59.896575 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:59.896534    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:15:59.921451 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:15:59.921397    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984" podStartSLOduration=10.921382819 podStartE2EDuration="10.921382819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:15:50.760587435 +0000 UTC m=+2287.253383611" watchObservedRunningTime="2026-07-10 21:15:59.921382819 +0000 UTC m=+2296.414178999"
Jul 10 21:16:00.803864 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:00.803832    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:16:27.713354 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:27.713322    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"]
Jul 10 21:16:42.735058 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:42.734941    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984" podUID="53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" containerName="main" containerID="cri-o://cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd" gracePeriod=45
Jul 10 21:16:42.974814 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:42.974789    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:16:43.028304 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028229    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-home\") pod \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") "
Jul 10 21:16:43.028304 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028278    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tmp-dir\") pod \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") "
Jul 10 21:16:43.028513 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028313    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-model-cache\") pod \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") "
Jul 10 21:16:43.028513 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028354    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkqnw\" (UniqueName: \"kubernetes.io/projected/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-kube-api-access-vkqnw\") pod \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") "
Jul 10 21:16:43.028513 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028407    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-dshm\") pod \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") "
Jul 10 21:16:43.028513 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028434    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tls-certs\") pod \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\" (UID: \"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0\") "
Jul 10 21:16:43.028752 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028548    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-home" (OuterVolumeSpecName: "home") pod "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" (UID: "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:16:43.028752 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028580    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" (UID: "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:16:43.028752 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028674    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-model-cache" (OuterVolumeSpecName: "model-cache") pod "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" (UID: "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:16:43.028877 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028780    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-home\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:16:43.028877 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028822    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tmp-dir\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:16:43.028877 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.028837    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-model-cache\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:16:43.030881 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.030848    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-dshm" (OuterVolumeSpecName: "dshm") pod "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" (UID: "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:16:43.030881 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.030854    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" (UID: "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 21:16:43.031018 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.030878    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-kube-api-access-vkqnw" (OuterVolumeSpecName: "kube-api-access-vkqnw") pod "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" (UID: "53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0"). InnerVolumeSpecName "kube-api-access-vkqnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:16:43.039371 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.039347    2565 generic.go:356] "Generic (PLEG): container finished" podID="53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" containerID="cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd" exitCode=0
Jul 10 21:16:43.039489 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.039381    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984" event={"ID":"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0","Type":"ContainerDied","Data":"cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd"}
Jul 10 21:16:43.039489 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.039399    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"
Jul 10 21:16:43.039489 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.039409    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984" event={"ID":"53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0","Type":"ContainerDied","Data":"9a07082fd5c785445f17bb343e7fb90d67c0465bb9746b24125e92f6592f46f6"}
Jul 10 21:16:43.039489 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.039430    2565 scope.go:117] "RemoveContainer" containerID="cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd"
Jul 10 21:16:43.051866 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.051846    2565 scope.go:117] "RemoveContainer" containerID="cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd"
Jul 10 21:16:43.052161 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:16:43.052143    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd\": container with ID starting with cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd not found: ID does not exist" containerID="cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd"
Jul 10 21:16:43.052209 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.052170    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd"} err="failed to get container status \"cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd\": rpc error: code = NotFound desc = could not find container \"cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd\": container with ID starting with cafb1120a22aa9650b691ead18cb8739d6bacf3d53ec47e91e6cb38ea3e199cd not found: ID does not exist"
Jul 10 21:16:43.064066 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.064045    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"]
Jul 10 21:16:43.068280 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.068258    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-46c4a127/e2e-lifecycle-model-46c4a127-kserve-7778d97859-6f984"]
Jul 10 21:16:43.129480 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.129448    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkqnw\" (UniqueName: \"kubernetes.io/projected/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-kube-api-access-vkqnw\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:16:43.129480 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.129475    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-dshm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:16:43.129480 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.129486    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0-tls-certs\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:16:43.958547 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:43.958503    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" path="/var/lib/kubelet/pods/53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0/volumes"
Jul 10 21:16:44.160020 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:16:44.159991    2565 scope.go:117] "RemoveContainer" containerID="8d0f173dd7c2e3d40373f8fd3a0b14ae384b562c8764105dc5701246e989f185"
Jul 10 21:17:44.372256 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:17:44.372127    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:17:44.378382 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:17:44.378347    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:17:44.381220 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:17:44.381190    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:17:44.386308 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:17:44.386292    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:18:44.194028 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:44.193998    2565 scope.go:117] "RemoveContainer" containerID="b53905982c7e01e1ae8bead796904ce4533fdd98d90c61b456802e67ab4cb355"
Jul 10 21:18:48.949071 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.949033    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"]
Jul 10 21:18:48.949472 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.949423    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" containerName="main"
Jul 10 21:18:48.949472 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.949435    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" containerName="main"
Jul 10 21:18:48.949582 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.949495    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="53b24fd4-7c2e-4ec6-aa02-c0e5fc2199f0" containerName="main"
Jul 10 21:18:48.953894 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.953878    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:48.956769 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.956747    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-42c194\"/\"default-dockercfg-zzxqw\""
Jul 10 21:18:48.957892 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.957862    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-42c194\"/\"kube-root-ca.crt\""
Jul 10 21:18:48.957892 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.957862    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-42c194\"/\"openshift-service-ca.crt\""
Jul 10 21:18:48.958073 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.957869    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-42c194\"/\"test-model-42c194-kserve-self-signed-certs\""
Jul 10 21:18:48.963082 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:48.963062    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"]
Jul 10 21:18:49.027791 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.027755    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqgb9\" (UniqueName: \"kubernetes.io/projected/30e75c1b-c596-4288-839f-09a539a080c3-kube-api-access-xqgb9\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.027791 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.027789    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-tmp-dir\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.028017 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.027861    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-dshm\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.028017 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.027889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30e75c1b-c596-4288-839f-09a539a080c3-tls-certs\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.028017 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.027945    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-model-cache\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.028017 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.027968    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-home\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.129363 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.129330    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqgb9\" (UniqueName: \"kubernetes.io/projected/30e75c1b-c596-4288-839f-09a539a080c3-kube-api-access-xqgb9\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.129363 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.129365    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-tmp-dir\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.129624 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.129415    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-dshm\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.129624 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.129430    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30e75c1b-c596-4288-839f-09a539a080c3-tls-certs\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.129624 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.129459    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-model-cache\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.129624 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.129479    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-home\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.129898 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.129861    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-tmp-dir\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.129996 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.129918    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-home\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.129996 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.129977    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-model-cache\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.131920 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.131900    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-dshm\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.132079 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.132062    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30e75c1b-c596-4288-839f-09a539a080c3-tls-certs\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.137705 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.137680    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqgb9\" (UniqueName: \"kubernetes.io/projected/30e75c1b-c596-4288-839f-09a539a080c3-kube-api-access-xqgb9\") pod \"test-model-42c194-kserve-7c9567f57f-qrq66\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") " pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.263480 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.263392    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:49.407043 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.407014    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"]
Jul 10 21:18:49.408762 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:18:49.408732    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30e75c1b_c596_4288_839f_09a539a080c3.slice/crio-463ec8349ddf64e61a36494fb5426bfdde9c460f52c9483115d882fd524e72f9 WatchSource:0}: Error finding container 463ec8349ddf64e61a36494fb5426bfdde9c460f52c9483115d882fd524e72f9: Status 404 returned error can't find the container with id 463ec8349ddf64e61a36494fb5426bfdde9c460f52c9483115d882fd524e72f9
Jul 10 21:18:49.745398 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.745353    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66" event={"ID":"30e75c1b-c596-4288-839f-09a539a080c3","Type":"ContainerStarted","Data":"537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35"}
Jul 10 21:18:49.745597 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.745408    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66" event={"ID":"30e75c1b-c596-4288-839f-09a539a080c3","Type":"ContainerStarted","Data":"463ec8349ddf64e61a36494fb5426bfdde9c460f52c9483115d882fd524e72f9"}
Jul 10 21:18:49.767079 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:49.767026    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66" podStartSLOduration=1.767010826 podStartE2EDuration="1.767010826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:18:49.764852049 +0000 UTC m=+2466.257648228" watchObservedRunningTime="2026-07-10 21:18:49.767010826 +0000 UTC m=+2466.259807005"
Jul 10 21:18:59.263734 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:59.263696    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:59.263734 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:59.263748    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:59.276174 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:59.276147    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:18:59.817458 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:18:59.817429    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:19:14.053053 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.053018    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"]
Jul 10 21:19:14.056797 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.056776    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.061203 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.061057    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5f5b2e\"/\"default-dockercfg-7862g\""
Jul 10 21:19:14.061203 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.061092    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5f5b2e\"/\"kube-root-ca.crt\""
Jul 10 21:19:14.061203 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.061066    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5f5b2e\"/\"test-model-5f5b2e-kserve-self-signed-certs\""
Jul 10 21:19:14.061450 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.061066    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5f5b2e\"/\"openshift-service-ca.crt\""
Jul 10 21:19:14.067656 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.067624    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"]
Jul 10 21:19:14.152936 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.152899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-tmp-dir\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.153125 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.152945    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv98z\" (UniqueName: \"kubernetes.io/projected/3ea83239-08d1-485d-a278-39fcde69f2f3-kube-api-access-xv98z\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.153125 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.153041    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-model-cache\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.153125 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.153068    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea83239-08d1-485d-a278-39fcde69f2f3-tls-certs\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.153300 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.153163    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-home\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.153300 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.153205    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-dshm\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.253936 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.253900    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv98z\" (UniqueName: \"kubernetes.io/projected/3ea83239-08d1-485d-a278-39fcde69f2f3-kube-api-access-xv98z\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.254158 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.253970    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-model-cache\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.254158 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.253989    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea83239-08d1-485d-a278-39fcde69f2f3-tls-certs\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.254158 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.254031    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-home\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.254158 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.254048    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-dshm\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.254158 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.254079    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-tmp-dir\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.254445 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.254419    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-model-cache\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.254445 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.254438    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-home\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.254618 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.254483    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-tmp-dir\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.256419 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.256403    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-dshm\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.256809 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.256793    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea83239-08d1-485d-a278-39fcde69f2f3-tls-certs\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.262470 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.262450    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv98z\" (UniqueName: \"kubernetes.io/projected/3ea83239-08d1-485d-a278-39fcde69f2f3-kube-api-access-xv98z\") pod \"test-model-5f5b2e-kserve-7bfd9564cf-5sdfw\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") " pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.367719 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.367644    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:14.503834 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.503804    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"]
Jul 10 21:19:14.505082 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:19:14.505049    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ea83239_08d1_485d_a278_39fcde69f2f3.slice/crio-ba7e7a9bdad5ed0de7ce613f30049474cfe9daf05ae765098a26fa85612e043a WatchSource:0}: Error finding container ba7e7a9bdad5ed0de7ce613f30049474cfe9daf05ae765098a26fa85612e043a: Status 404 returned error can't find the container with id ba7e7a9bdad5ed0de7ce613f30049474cfe9daf05ae765098a26fa85612e043a
Jul 10 21:19:14.890004 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.889957    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw" event={"ID":"3ea83239-08d1-485d-a278-39fcde69f2f3","Type":"ContainerStarted","Data":"79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba"}
Jul 10 21:19:14.890004 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.890007    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw" event={"ID":"3ea83239-08d1-485d-a278-39fcde69f2f3","Type":"ContainerStarted","Data":"ba7e7a9bdad5ed0de7ce613f30049474cfe9daf05ae765098a26fa85612e043a"}
Jul 10 21:19:14.908175 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:14.908125    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw" podStartSLOduration=0.908110611 podStartE2EDuration="908.110611ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:19:14.906370618 +0000 UTC m=+2491.399166821" watchObservedRunningTime="2026-07-10 21:19:14.908110611 +0000 UTC m=+2491.400906789"
Jul 10 21:19:24.368757 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:24.368714    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:24.369332 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:24.368779    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:24.382785 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:24.382761    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:24.956054 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:24.956020    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:34.153654 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:34.153618    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"]
Jul 10 21:19:35.144459 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:35.144426    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"]
Jul 10 21:19:49.174441 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.174371    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66" podUID="30e75c1b-c596-4288-839f-09a539a080c3" containerName="main" containerID="cri-o://537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35" gracePeriod=45
Jul 10 21:19:49.427251 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.427186    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:19:49.486012 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.485982    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-tmp-dir\") pod \"30e75c1b-c596-4288-839f-09a539a080c3\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") "
Jul 10 21:19:49.486165 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486032    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-model-cache\") pod \"30e75c1b-c596-4288-839f-09a539a080c3\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") "
Jul 10 21:19:49.486165 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486068    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqgb9\" (UniqueName: \"kubernetes.io/projected/30e75c1b-c596-4288-839f-09a539a080c3-kube-api-access-xqgb9\") pod \"30e75c1b-c596-4288-839f-09a539a080c3\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") "
Jul 10 21:19:49.486235 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486163    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-dshm\") pod \"30e75c1b-c596-4288-839f-09a539a080c3\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") "
Jul 10 21:19:49.486235 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486187    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30e75c1b-c596-4288-839f-09a539a080c3-tls-certs\") pod \"30e75c1b-c596-4288-839f-09a539a080c3\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") "
Jul 10 21:19:49.486235 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486221    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-home\") pod \"30e75c1b-c596-4288-839f-09a539a080c3\" (UID: \"30e75c1b-c596-4288-839f-09a539a080c3\") "
Jul 10 21:19:49.486393 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486279    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "30e75c1b-c596-4288-839f-09a539a080c3" (UID: "30e75c1b-c596-4288-839f-09a539a080c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:19:49.486393 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486367    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "30e75c1b-c596-4288-839f-09a539a080c3" (UID: "30e75c1b-c596-4288-839f-09a539a080c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:19:49.486507 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486487    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-home" (OuterVolumeSpecName: "home") pod "30e75c1b-c596-4288-839f-09a539a080c3" (UID: "30e75c1b-c596-4288-839f-09a539a080c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:19:49.486600 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486553    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-tmp-dir\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:49.486600 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.486589    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-model-cache\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:49.488516 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.488488    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-dshm" (OuterVolumeSpecName: "dshm") pod "30e75c1b-c596-4288-839f-09a539a080c3" (UID: "30e75c1b-c596-4288-839f-09a539a080c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:19:49.488664 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.488531    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e75c1b-c596-4288-839f-09a539a080c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "30e75c1b-c596-4288-839f-09a539a080c3" (UID: "30e75c1b-c596-4288-839f-09a539a080c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 21:19:49.488664 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.488653    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30e75c1b-c596-4288-839f-09a539a080c3-kube-api-access-xqgb9" (OuterVolumeSpecName: "kube-api-access-xqgb9") pod "30e75c1b-c596-4288-839f-09a539a080c3" (UID: "30e75c1b-c596-4288-839f-09a539a080c3"). InnerVolumeSpecName "kube-api-access-xqgb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:19:49.587924 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.587877    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqgb9\" (UniqueName: \"kubernetes.io/projected/30e75c1b-c596-4288-839f-09a539a080c3-kube-api-access-xqgb9\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:49.587924 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.587916    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-dshm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:49.587924 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.587926    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30e75c1b-c596-4288-839f-09a539a080c3-tls-certs\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:49.587924 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:49.587935    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30e75c1b-c596-4288-839f-09a539a080c3-home\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:50.081104 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.081067    2565 generic.go:356] "Generic (PLEG): container finished" podID="30e75c1b-c596-4288-839f-09a539a080c3" containerID="537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35" exitCode=0
Jul 10 21:19:50.081284 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.081109    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"
Jul 10 21:19:50.081284 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.081151    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66" event={"ID":"30e75c1b-c596-4288-839f-09a539a080c3","Type":"ContainerDied","Data":"537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35"}
Jul 10 21:19:50.081284 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.081189    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66" event={"ID":"30e75c1b-c596-4288-839f-09a539a080c3","Type":"ContainerDied","Data":"463ec8349ddf64e61a36494fb5426bfdde9c460f52c9483115d882fd524e72f9"}
Jul 10 21:19:50.081284 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.081207    2565 scope.go:117] "RemoveContainer" containerID="537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35"
Jul 10 21:19:50.090733 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.090713    2565 scope.go:117] "RemoveContainer" containerID="537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35"
Jul 10 21:19:50.090999 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:19:50.090976    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35\": container with ID starting with 537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35 not found: ID does not exist" containerID="537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35"
Jul 10 21:19:50.091108 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.091015    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35"} err="failed to get container status \"537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35\": rpc error: code = NotFound desc = could not find container \"537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35\": container with ID starting with 537888f7dda9d031d1a854dee52b5fc1b96ead001fd2d51771dc57c8f89eaa35 not found: ID does not exist"
Jul 10 21:19:50.099407 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.099381    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"]
Jul 10 21:19:50.102982 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.102957    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-42c194/test-model-42c194-kserve-7c9567f57f-qrq66"]
Jul 10 21:19:50.165636 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.165596    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw" podUID="3ea83239-08d1-485d-a278-39fcde69f2f3" containerName="main" containerID="cri-o://79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba" gracePeriod=45
Jul 10 21:19:50.420792 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.420770    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:50.496183 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496104    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-model-cache\") pod \"3ea83239-08d1-485d-a278-39fcde69f2f3\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") "
Jul 10 21:19:50.496183 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496141    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-tmp-dir\") pod \"3ea83239-08d1-485d-a278-39fcde69f2f3\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") "
Jul 10 21:19:50.496183 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496182    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv98z\" (UniqueName: \"kubernetes.io/projected/3ea83239-08d1-485d-a278-39fcde69f2f3-kube-api-access-xv98z\") pod \"3ea83239-08d1-485d-a278-39fcde69f2f3\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") "
Jul 10 21:19:50.496420 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496243    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-home\") pod \"3ea83239-08d1-485d-a278-39fcde69f2f3\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") "
Jul 10 21:19:50.496420 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496261    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-dshm\") pod \"3ea83239-08d1-485d-a278-39fcde69f2f3\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") "
Jul 10 21:19:50.496420 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496284    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea83239-08d1-485d-a278-39fcde69f2f3-tls-certs\") pod \"3ea83239-08d1-485d-a278-39fcde69f2f3\" (UID: \"3ea83239-08d1-485d-a278-39fcde69f2f3\") "
Jul 10 21:19:50.496610 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496420    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ea83239-08d1-485d-a278-39fcde69f2f3" (UID: "3ea83239-08d1-485d-a278-39fcde69f2f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:19:50.496610 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496442    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3ea83239-08d1-485d-a278-39fcde69f2f3" (UID: "3ea83239-08d1-485d-a278-39fcde69f2f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:19:50.496610 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496523    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-home" (OuterVolumeSpecName: "home") pod "3ea83239-08d1-485d-a278-39fcde69f2f3" (UID: "3ea83239-08d1-485d-a278-39fcde69f2f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:19:50.496610 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496596    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-model-cache\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:50.496610 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.496611    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-tmp-dir\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:50.498684 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.498658    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ea83239-08d1-485d-a278-39fcde69f2f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ea83239-08d1-485d-a278-39fcde69f2f3" (UID: "3ea83239-08d1-485d-a278-39fcde69f2f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 21:19:50.498771 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.498667    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-dshm" (OuterVolumeSpecName: "dshm") pod "3ea83239-08d1-485d-a278-39fcde69f2f3" (UID: "3ea83239-08d1-485d-a278-39fcde69f2f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:19:50.498771 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.498721    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ea83239-08d1-485d-a278-39fcde69f2f3-kube-api-access-xv98z" (OuterVolumeSpecName: "kube-api-access-xv98z") pod "3ea83239-08d1-485d-a278-39fcde69f2f3" (UID: "3ea83239-08d1-485d-a278-39fcde69f2f3"). InnerVolumeSpecName "kube-api-access-xv98z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:19:50.597942 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.597903    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv98z\" (UniqueName: \"kubernetes.io/projected/3ea83239-08d1-485d-a278-39fcde69f2f3-kube-api-access-xv98z\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:50.597942 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.597946    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-home\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:50.598148 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.597961    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ea83239-08d1-485d-a278-39fcde69f2f3-dshm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:50.598148 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:50.597974    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea83239-08d1-485d-a278-39fcde69f2f3-tls-certs\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:19:51.088839 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.088802    2565 generic.go:356] "Generic (PLEG): container finished" podID="3ea83239-08d1-485d-a278-39fcde69f2f3" containerID="79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba" exitCode=0
Jul 10 21:19:51.088839 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.088842    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"
Jul 10 21:19:51.089075 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.088885    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw" event={"ID":"3ea83239-08d1-485d-a278-39fcde69f2f3","Type":"ContainerDied","Data":"79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba"}
Jul 10 21:19:51.089075 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.088923    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw" event={"ID":"3ea83239-08d1-485d-a278-39fcde69f2f3","Type":"ContainerDied","Data":"ba7e7a9bdad5ed0de7ce613f30049474cfe9daf05ae765098a26fa85612e043a"}
Jul 10 21:19:51.089075 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.088942    2565 scope.go:117] "RemoveContainer" containerID="79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba"
Jul 10 21:19:51.098312 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.098293    2565 scope.go:117] "RemoveContainer" containerID="79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba"
Jul 10 21:19:51.098541 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:19:51.098524    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba\": container with ID starting with 79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba not found: ID does not exist" containerID="79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba"
Jul 10 21:19:51.098619 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.098549    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba"} err="failed to get container status \"79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba\": rpc error: code = NotFound desc = could not find container \"79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba\": container with ID starting with 79782c9442dbf512a838e74d1c33b36e32d742040c9dc67a10c10a500dee64ba not found: ID does not exist"
Jul 10 21:19:51.110170 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.110146    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"]
Jul 10 21:19:51.114267 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.114242    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-5f5b2e/test-model-5f5b2e-kserve-7bfd9564cf-5sdfw"]
Jul 10 21:19:51.957993 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.957905    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30e75c1b-c596-4288-839f-09a539a080c3" path="/var/lib/kubelet/pods/30e75c1b-c596-4288-839f-09a539a080c3/volumes"
Jul 10 21:19:51.958476 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:19:51.958455    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ea83239-08d1-485d-a278-39fcde69f2f3" path="/var/lib/kubelet/pods/3ea83239-08d1-485d-a278-39fcde69f2f3/volumes"
Jul 10 21:20:01.524890 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.524853    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"]
Jul 10 21:20:01.525338 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.525238    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30e75c1b-c596-4288-839f-09a539a080c3" containerName="main"
Jul 10 21:20:01.525338 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.525249    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="30e75c1b-c596-4288-839f-09a539a080c3" containerName="main"
Jul 10 21:20:01.525338 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.525265    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ea83239-08d1-485d-a278-39fcde69f2f3" containerName="main"
Jul 10 21:20:01.525338 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.525270    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ea83239-08d1-485d-a278-39fcde69f2f3" containerName="main"
Jul 10 21:20:01.525338 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.525322    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="30e75c1b-c596-4288-839f-09a539a080c3" containerName="main"
Jul 10 21:20:01.525338 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.525331    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ea83239-08d1-485d-a278-39fcde69f2f3" containerName="main"
Jul 10 21:20:01.529691 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.529672    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"
Jul 10 21:20:01.532785 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.532759    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 21:20:01.532922 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.532908    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:20:01.549056 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.549031    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"]
Jul 10 21:20:01.592362 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.592328    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec122bc-4030-4deb-a098-6ab8aaec43d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp\" (UID: \"aec122bc-4030-4deb-a098-6ab8aaec43d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"
Jul 10 21:20:01.692937 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.692896    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec122bc-4030-4deb-a098-6ab8aaec43d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp\" (UID: \"aec122bc-4030-4deb-a098-6ab8aaec43d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"
Jul 10 21:20:01.693545 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.693525    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec122bc-4030-4deb-a098-6ab8aaec43d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp\" (UID: \"aec122bc-4030-4deb-a098-6ab8aaec43d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"
Jul 10 21:20:01.839797 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.839763    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"
Jul 10 21:20:01.986070 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:01.986037    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"]
Jul 10 21:20:01.987523 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:20:01.987503    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec122bc_4030_4deb_a098_6ab8aaec43d3.slice/crio-adc9b4f45d39d4db66bdf596e52147507f70fc1b9f0de8bcca90637ca888013a WatchSource:0}: Error finding container adc9b4f45d39d4db66bdf596e52147507f70fc1b9f0de8bcca90637ca888013a: Status 404 returned error can't find the container with id adc9b4f45d39d4db66bdf596e52147507f70fc1b9f0de8bcca90637ca888013a
Jul 10 21:20:02.154191 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:02.154106    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp" event={"ID":"aec122bc-4030-4deb-a098-6ab8aaec43d3","Type":"ContainerStarted","Data":"adc9b4f45d39d4db66bdf596e52147507f70fc1b9f0de8bcca90637ca888013a"}
Jul 10 21:20:03.160966 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:03.160872    2565 generic.go:356] "Generic (PLEG): container finished" podID="aec122bc-4030-4deb-a098-6ab8aaec43d3" containerID="b7ddbca48aa98409b9545b811de8e74cd14638e2c63253323ac526a2ddd6a498" exitCode=0
Jul 10 21:20:03.160966 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:03.160954    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp" event={"ID":"aec122bc-4030-4deb-a098-6ab8aaec43d3","Type":"ContainerDied","Data":"b7ddbca48aa98409b9545b811de8e74cd14638e2c63253323ac526a2ddd6a498"}
Jul 10 21:20:04.295932 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:04.295908    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"
Jul 10 21:20:04.417180 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:04.417097    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec122bc-4030-4deb-a098-6ab8aaec43d3-maas-api-service-ca\") pod \"aec122bc-4030-4deb-a098-6ab8aaec43d3\" (UID: \"aec122bc-4030-4deb-a098-6ab8aaec43d3\") "
Jul 10 21:20:04.417439 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:04.417417    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aec122bc-4030-4deb-a098-6ab8aaec43d3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aec122bc-4030-4deb-a098-6ab8aaec43d3" (UID: "aec122bc-4030-4deb-a098-6ab8aaec43d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 21:20:04.518338 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:04.518303    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec122bc-4030-4deb-a098-6ab8aaec43d3-maas-api-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:20:05.172949 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:05.172908    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp" event={"ID":"aec122bc-4030-4deb-a098-6ab8aaec43d3","Type":"ContainerDied","Data":"adc9b4f45d39d4db66bdf596e52147507f70fc1b9f0de8bcca90637ca888013a"}
Jul 10 21:20:05.172949 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:05.172938    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp"
Jul 10 21:20:05.173158 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:05.172967    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adc9b4f45d39d4db66bdf596e52147507f70fc1b9f0de8bcca90637ca888013a"
Jul 10 21:20:52.224520 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.224477    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"]
Jul 10 21:20:52.225105 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.225084    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec122bc-4030-4deb-a098-6ab8aaec43d3" containerName="revoke-keys"
Jul 10 21:20:52.225192 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.225108    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec122bc-4030-4deb-a098-6ab8aaec43d3" containerName="revoke-keys"
Jul 10 21:20:52.225249 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.225219    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec122bc-4030-4deb-a098-6ab8aaec43d3" containerName="revoke-keys"
Jul 10 21:20:52.228250 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.228228    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"
Jul 10 21:20:52.232043 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.232016    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 21:20:52.232292 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.232270    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:20:52.234543 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.234516    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"]
Jul 10 21:20:52.351719 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.351682    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw\" (UID: \"5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"
Jul 10 21:20:52.452955 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.452921    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw\" (UID: \"5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"
Jul 10 21:20:52.453574 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.453539    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw\" (UID: \"5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"
Jul 10 21:20:52.541755 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.541675    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"
Jul 10 21:20:52.689370 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:52.689339    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"]
Jul 10 21:20:52.690801 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:20:52.690775    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e11a7e4_9e8e_48c2_b9bb_54d4593a3c3f.slice/crio-02303ce4f6f640a6d65d5b55acf941713e8c09333dbb9de0be04d7ccd1f58ab1 WatchSource:0}: Error finding container 02303ce4f6f640a6d65d5b55acf941713e8c09333dbb9de0be04d7ccd1f58ab1: Status 404 returned error can't find the container with id 02303ce4f6f640a6d65d5b55acf941713e8c09333dbb9de0be04d7ccd1f58ab1
Jul 10 21:20:53.445054 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:53.445018    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw" event={"ID":"5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f","Type":"ContainerStarted","Data":"336e9c9c92fa51db104a815bb3f8ff78092714326b8e28def1df9e191bcd792d"}
Jul 10 21:20:53.445402 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:53.445079    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw" event={"ID":"5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f","Type":"ContainerStarted","Data":"02303ce4f6f640a6d65d5b55acf941713e8c09333dbb9de0be04d7ccd1f58ab1"}
Jul 10 21:20:53.461667 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:53.461601    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw" podStartSLOduration=1.461585576 podStartE2EDuration="1.461585576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:20:53.460365613 +0000 UTC m=+2589.953161795" watchObservedRunningTime="2026-07-10 21:20:53.461585576 +0000 UTC m=+2589.954381746"
Jul 10 21:20:54.452598 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:54.452546    2565 generic.go:356] "Generic (PLEG): container finished" podID="5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f" containerID="336e9c9c92fa51db104a815bb3f8ff78092714326b8e28def1df9e191bcd792d" exitCode=0
Jul 10 21:20:54.453004 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:54.452591    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw" event={"ID":"5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f","Type":"ContainerDied","Data":"336e9c9c92fa51db104a815bb3f8ff78092714326b8e28def1df9e191bcd792d"}
Jul 10 21:20:55.588930 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:55.588902    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"
Jul 10 21:20:55.689924 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:55.689894    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f-maas-api-service-ca\") pod \"5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f\" (UID: \"5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f\") "
Jul 10 21:20:55.690298 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:55.690264    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f" (UID: "5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 21:20:55.791443 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:55.791364    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f-maas-api-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:20:56.463916 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:56.463883    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw"
Jul 10 21:20:56.464112 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:56.463923    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw" event={"ID":"5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f","Type":"ContainerDied","Data":"02303ce4f6f640a6d65d5b55acf941713e8c09333dbb9de0be04d7ccd1f58ab1"}
Jul 10 21:20:56.464112 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:20:56.463972    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02303ce4f6f640a6d65d5b55acf941713e8c09333dbb9de0be04d7ccd1f58ab1"
Jul 10 21:21:18.436855 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.436775    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"]
Jul 10 21:21:18.437236 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.437179    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f" containerName="revoke-keys"
Jul 10 21:21:18.437236 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.437195    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f" containerName="revoke-keys"
Jul 10 21:21:18.437308 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.437280    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f" containerName="revoke-keys"
Jul 10 21:21:18.440538 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.440523    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"
Jul 10 21:21:18.443102 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.443078    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 21:21:18.443237 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.443190    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:21:18.447496 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.447470    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"]
Jul 10 21:21:18.487023 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.486987    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e416369d-513e-4c1c-aed5-b71ad1b36d52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq\" (UID: \"e416369d-513e-4c1c-aed5-b71ad1b36d52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"
Jul 10 21:21:18.588074 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.588041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e416369d-513e-4c1c-aed5-b71ad1b36d52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq\" (UID: \"e416369d-513e-4c1c-aed5-b71ad1b36d52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"
Jul 10 21:21:18.588647 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.588628    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e416369d-513e-4c1c-aed5-b71ad1b36d52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq\" (UID: \"e416369d-513e-4c1c-aed5-b71ad1b36d52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"
Jul 10 21:21:18.751981 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.751898    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"
Jul 10 21:21:18.884668 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:18.884639    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"]
Jul 10 21:21:18.886318 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:21:18.886292    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode416369d_513e_4c1c_aed5_b71ad1b36d52.slice/crio-fb7e42da97477903242569eac00e1a141eaa13e29b2689f0a23ceb0bab0bba7b WatchSource:0}: Error finding container fb7e42da97477903242569eac00e1a141eaa13e29b2689f0a23ceb0bab0bba7b: Status 404 returned error can't find the container with id fb7e42da97477903242569eac00e1a141eaa13e29b2689f0a23ceb0bab0bba7b
Jul 10 21:21:19.595889 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:19.595850    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq" event={"ID":"e416369d-513e-4c1c-aed5-b71ad1b36d52","Type":"ContainerStarted","Data":"b1704b0a4ab78994560c6239dadb82d0691432c68201f82a2895313004529d7a"}
Jul 10 21:21:19.596275 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:19.595902    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq" event={"ID":"e416369d-513e-4c1c-aed5-b71ad1b36d52","Type":"ContainerStarted","Data":"fb7e42da97477903242569eac00e1a141eaa13e29b2689f0a23ceb0bab0bba7b"}
Jul 10 21:21:19.614210 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:19.614168    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq" podStartSLOduration=1.614154662 podStartE2EDuration="1.614154662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:21:19.611083292 +0000 UTC m=+2616.103879470" watchObservedRunningTime="2026-07-10 21:21:19.614154662 +0000 UTC m=+2616.106950841"
Jul 10 21:21:20.602515 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:20.602478    2565 generic.go:356] "Generic (PLEG): container finished" podID="e416369d-513e-4c1c-aed5-b71ad1b36d52" containerID="b1704b0a4ab78994560c6239dadb82d0691432c68201f82a2895313004529d7a" exitCode=0
Jul 10 21:21:20.602903 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:20.602589    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq" event={"ID":"e416369d-513e-4c1c-aed5-b71ad1b36d52","Type":"ContainerDied","Data":"b1704b0a4ab78994560c6239dadb82d0691432c68201f82a2895313004529d7a"}
Jul 10 21:21:21.742163 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:21.742138    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"
Jul 10 21:21:21.815851 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:21.815819    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e416369d-513e-4c1c-aed5-b71ad1b36d52-maas-api-service-ca\") pod \"e416369d-513e-4c1c-aed5-b71ad1b36d52\" (UID: \"e416369d-513e-4c1c-aed5-b71ad1b36d52\") "
Jul 10 21:21:21.816174 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:21.816151    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e416369d-513e-4c1c-aed5-b71ad1b36d52-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e416369d-513e-4c1c-aed5-b71ad1b36d52" (UID: "e416369d-513e-4c1c-aed5-b71ad1b36d52"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 21:21:21.917439 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:21.917353    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e416369d-513e-4c1c-aed5-b71ad1b36d52-maas-api-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:21:22.615302 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:22.615271    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq"
Jul 10 21:21:22.615473 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:22.615297    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq" event={"ID":"e416369d-513e-4c1c-aed5-b71ad1b36d52","Type":"ContainerDied","Data":"fb7e42da97477903242569eac00e1a141eaa13e29b2689f0a23ceb0bab0bba7b"}
Jul 10 21:21:22.615473 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:21:22.615363    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb7e42da97477903242569eac00e1a141eaa13e29b2689f0a23ceb0bab0bba7b"
Jul 10 21:22:18.557519 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.557481    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"]
Jul 10 21:22:18.557974 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.557863    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e416369d-513e-4c1c-aed5-b71ad1b36d52" containerName="revoke-keys"
Jul 10 21:22:18.557974 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.557875    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e416369d-513e-4c1c-aed5-b71ad1b36d52" containerName="revoke-keys"
Jul 10 21:22:18.557974 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.557958    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e416369d-513e-4c1c-aed5-b71ad1b36d52" containerName="revoke-keys"
Jul 10 21:22:18.561077 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.561060    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.565757 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.565727    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-920ed7\"/\"default-dockercfg-h4dnh\""
Jul 10 21:22:18.565757 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.565740    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-920ed7\"/\"openshift-service-ca.crt\""
Jul 10 21:22:18.566015 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.565732    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-920ed7\"/\"kube-root-ca.crt\""
Jul 10 21:22:18.566015 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.565786    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-920ed7\"/\"auth-test-model-920ed7-kserve-self-signed-certs\""
Jul 10 21:22:18.572430 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.572407    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"]
Jul 10 21:22:18.633788 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.633759    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-home\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.633788 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.633795    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tmp-dir\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.634025 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.633815    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd6n5\" (UniqueName: \"kubernetes.io/projected/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-kube-api-access-vd6n5\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.634025 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.633920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tls-certs\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.634025 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.633959    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-dshm\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.634167 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.634027    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-model-cache\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.734592 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.734534    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-home\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.734592 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.734606    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tmp-dir\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.734846 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.734631    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd6n5\" (UniqueName: \"kubernetes.io/projected/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-kube-api-access-vd6n5\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.734846 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.734833    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tls-certs\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.734961 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.734870    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-dshm\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.734961 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.734915    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-model-cache\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.735068 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.734976    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-home\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.735068 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.735013    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tmp-dir\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.735252 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.735229    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-model-cache\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.737243 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.737222    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-dshm\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.737448 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.737434    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tls-certs\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.743580 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.743535    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd6n5\" (UniqueName: \"kubernetes.io/projected/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-kube-api-access-vd6n5\") pod \"auth-test-model-920ed7-kserve-5d9d48bd7-f884h\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") " pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:18.870892 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:18.870806    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:19.011244 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:19.011214    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"]
Jul 10 21:22:19.013152 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:22:19.013125    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd6a6a0e_147d_4470_8c0f_7cfd5a38a594.slice/crio-814a532c520eb8e4b1019e9189314e6cd7e8fe9c99eeea49c4245ba47aa19a20 WatchSource:0}: Error finding container 814a532c520eb8e4b1019e9189314e6cd7e8fe9c99eeea49c4245ba47aa19a20: Status 404 returned error can't find the container with id 814a532c520eb8e4b1019e9189314e6cd7e8fe9c99eeea49c4245ba47aa19a20
Jul 10 21:22:19.936863 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:19.936821    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h" event={"ID":"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594","Type":"ContainerStarted","Data":"3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251"}
Jul 10 21:22:19.936863 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:19.936867    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h" event={"ID":"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594","Type":"ContainerStarted","Data":"814a532c520eb8e4b1019e9189314e6cd7e8fe9c99eeea49c4245ba47aa19a20"}
Jul 10 21:22:19.957766 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:19.957715    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h" podStartSLOduration=1.957699641 podStartE2EDuration="1.957699641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:22:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:22:19.95676988 +0000 UTC m=+2676.449566059" watchObservedRunningTime="2026-07-10 21:22:19.957699641 +0000 UTC m=+2676.450495832"
Jul 10 21:22:28.871851 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:28.871808    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:28.871851 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:28.871857    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:28.884361 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:28.884338    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:28.998429 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:28.998403    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:22:44.415324 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:44.415211    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:22:44.429032 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:44.420855    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:22:44.429032 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:44.422263    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:22:44.429032 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:44.427540    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:22:45.489755 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.489724    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"]
Jul 10 21:22:45.493157 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.493141    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.497775 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.497755    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"kube-root-ca.crt\""
Jul 10 21:22:45.497902 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.497755    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"auth-test-model-6e676e-kserve-self-signed-certs\""
Jul 10 21:22:45.497962 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.497941    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"default-dockercfg-mz5st\""
Jul 10 21:22:45.498042 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.498028    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"openshift-service-ca.crt\""
Jul 10 21:22:45.507691 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.507666    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"]
Jul 10 21:22:45.590368 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.590331    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m52m\" (UniqueName: \"kubernetes.io/projected/18ff8b39-2fff-4781-8f00-1c4bda2c7591-kube-api-access-2m52m\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.590368 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.590369    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-dshm\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.590626 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.590447    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-home\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.590626 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.590488    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-model-cache\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.590626 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.590512    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tls-certs\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.590626 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.590535    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tmp-dir\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.691371 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.691337    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-model-cache\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.691371 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.691373    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tls-certs\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.691611 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.691395    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tmp-dir\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.691611 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.691444    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m52m\" (UniqueName: \"kubernetes.io/projected/18ff8b39-2fff-4781-8f00-1c4bda2c7591-kube-api-access-2m52m\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.691611 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.691465    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-dshm\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.691611 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.691507    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-home\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.691845 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.691818    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-model-cache\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.691910 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.691857    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tmp-dir\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.691963 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.691908    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-home\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.693930 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.693908    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-dshm\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.694014 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.694001    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tls-certs\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.700768 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.700746    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m52m\" (UniqueName: \"kubernetes.io/projected/18ff8b39-2fff-4781-8f00-1c4bda2c7591-kube-api-access-2m52m\") pod \"auth-test-model-6e676e-kserve-5ff569695-48xvc\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") " pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.802711 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.802686    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:45.940637 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:45.940612    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"]
Jul 10 21:22:45.943041 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:22:45.943016    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18ff8b39_2fff_4781_8f00_1c4bda2c7591.slice/crio-cafb6d648d1ff1c2cbd64de02d272f3441ca26e40ead40ca6d9e3414db7626dd WatchSource:0}: Error finding container cafb6d648d1ff1c2cbd64de02d272f3441ca26e40ead40ca6d9e3414db7626dd: Status 404 returned error can't find the container with id cafb6d648d1ff1c2cbd64de02d272f3441ca26e40ead40ca6d9e3414db7626dd
Jul 10 21:22:46.084746 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:46.084660    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc" event={"ID":"18ff8b39-2fff-4781-8f00-1c4bda2c7591","Type":"ContainerStarted","Data":"5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21"}
Jul 10 21:22:46.084746 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:46.084711    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc" event={"ID":"18ff8b39-2fff-4781-8f00-1c4bda2c7591","Type":"ContainerStarted","Data":"cafb6d648d1ff1c2cbd64de02d272f3441ca26e40ead40ca6d9e3414db7626dd"}
Jul 10 21:22:46.111379 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:46.111312    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc" podStartSLOduration=1.111297197 podStartE2EDuration="1.111297197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:22:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:22:46.111129209 +0000 UTC m=+2702.603925388" watchObservedRunningTime="2026-07-10 21:22:46.111297197 +0000 UTC m=+2702.604093412"
Jul 10 21:22:55.802908 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:55.802872    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:55.803349 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:55.802924    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:55.815545 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:55.815524    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:22:56.151578 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:22:56.151477    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:24:42.738305 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:42.738269    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"]
Jul 10 21:24:43.569969 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:43.569934    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"]
Jul 10 21:24:53.164480 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.164447    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"]
Jul 10 21:24:53.167946 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.167928    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.170730 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.170705    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-920ed7\"/\"sub-test-model-920ed7-kserve-self-signed-certs\""
Jul 10 21:24:53.195632 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.195598    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"]
Jul 10 21:24:53.349020 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.348979    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-tmp-dir\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.349234 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.349088    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-model-cache\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.349234 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.349128    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff5rw\" (UniqueName: \"kubernetes.io/projected/05874faf-e245-4b64-a3e3-b2190421a73c-kube-api-access-ff5rw\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.349234 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.349166    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05874faf-e245-4b64-a3e3-b2190421a73c-tls-certs\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.349234 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.349206    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-home\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.349474 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.349256    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-dshm\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.450218 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.450105    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-home\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.450218 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.450176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-dshm\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.450218 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.450219    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-tmp-dir\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.450506 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.450296    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-model-cache\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.450506 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.450330    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff5rw\" (UniqueName: \"kubernetes.io/projected/05874faf-e245-4b64-a3e3-b2190421a73c-kube-api-access-ff5rw\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.450506 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.450364    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05874faf-e245-4b64-a3e3-b2190421a73c-tls-certs\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.450714 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.450543    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-home\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.450714 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.450695    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-model-cache\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.450827 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.450724    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-tmp-dir\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.452587 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.452531    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-dshm\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.452880 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.452863    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05874faf-e245-4b64-a3e3-b2190421a73c-tls-certs\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.459126 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.459098    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff5rw\" (UniqueName: \"kubernetes.io/projected/05874faf-e245-4b64-a3e3-b2190421a73c-kube-api-access-ff5rw\") pod \"sub-test-model-920ed7-kserve-6544778d9c-zkmsv\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") " pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.477184 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.477146    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:24:53.619319 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.619293    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"]
Jul 10 21:24:53.621089 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:24:53.621062    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05874faf_e245_4b64_a3e3_b2190421a73c.slice/crio-7c54754ffc2ae71ee7c5ed3dea48ea4bc74cfc8581f8ca9853d440f6788b4c19 WatchSource:0}: Error finding container 7c54754ffc2ae71ee7c5ed3dea48ea4bc74cfc8581f8ca9853d440f6788b4c19: Status 404 returned error can't find the container with id 7c54754ffc2ae71ee7c5ed3dea48ea4bc74cfc8581f8ca9853d440f6788b4c19
Jul 10 21:24:53.820489 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.820455    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv" event={"ID":"05874faf-e245-4b64-a3e3-b2190421a73c","Type":"ContainerStarted","Data":"5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198"}
Jul 10 21:24:53.820702 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.820496    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv" event={"ID":"05874faf-e245-4b64-a3e3-b2190421a73c","Type":"ContainerStarted","Data":"7c54754ffc2ae71ee7c5ed3dea48ea4bc74cfc8581f8ca9853d440f6788b4c19"}
Jul 10 21:24:53.839895 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:53.839841    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv" podStartSLOduration=0.839825722 podStartE2EDuration="839.825722ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:24:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:24:53.837794289 +0000 UTC m=+2830.330590470" watchObservedRunningTime="2026-07-10 21:24:53.839825722 +0000 UTC m=+2830.332621960"
Jul 10 21:24:57.760280 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:57.760237    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h" podUID="dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" containerName="main" containerID="cri-o://3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251" gracePeriod=45
Jul 10 21:24:58.025791 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.025726    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:24:58.096355 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.096312    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tmp-dir\") pod \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") "
Jul 10 21:24:58.096355 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.096360    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vd6n5\" (UniqueName: \"kubernetes.io/projected/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-kube-api-access-vd6n5\") pod \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") "
Jul 10 21:24:58.096698 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.096380    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-dshm\") pod \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") "
Jul 10 21:24:58.096698 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.096493    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tls-certs\") pod \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") "
Jul 10 21:24:58.096698 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.096608    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-home\") pod \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") "
Jul 10 21:24:58.096698 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.096642    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-model-cache\") pod \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\" (UID: \"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594\") "
Jul 10 21:24:58.096698 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.096647    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" (UID: "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:24:58.097056 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.097005    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tmp-dir\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:58.097056 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.097003    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-home" (OuterVolumeSpecName: "home") pod "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" (UID: "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:24:58.097414 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.097150    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" (UID: "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:24:58.098862 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.098839    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-dshm" (OuterVolumeSpecName: "dshm") pod "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" (UID: "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:24:58.099246 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.099222    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" (UID: "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 21:24:58.099340 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.099318    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-kube-api-access-vd6n5" (OuterVolumeSpecName: "kube-api-access-vd6n5") pod "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" (UID: "dd6a6a0e-147d-4470-8c0f-7cfd5a38a594"). InnerVolumeSpecName "kube-api-access-vd6n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:24:58.197619 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.197581    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-model-cache\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:58.197619 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.197611    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd6n5\" (UniqueName: \"kubernetes.io/projected/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-kube-api-access-vd6n5\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:58.197619 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.197621    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-dshm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:58.197844 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.197630    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-tls-certs\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:58.197844 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.197640    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594-home\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:58.590515 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.590466    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc" podUID="18ff8b39-2fff-4781-8f00-1c4bda2c7591" containerName="main" containerID="cri-o://5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21" gracePeriod=45
Jul 10 21:24:58.849185 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.849121    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:24:58.852540 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.852514    2565 generic.go:356] "Generic (PLEG): container finished" podID="dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" containerID="3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251" exitCode=0
Jul 10 21:24:58.852673 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.852552    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"
Jul 10 21:24:58.852673 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.852601    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h" event={"ID":"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594","Type":"ContainerDied","Data":"3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251"}
Jul 10 21:24:58.852673 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.852646    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h" event={"ID":"dd6a6a0e-147d-4470-8c0f-7cfd5a38a594","Type":"ContainerDied","Data":"814a532c520eb8e4b1019e9189314e6cd7e8fe9c99eeea49c4245ba47aa19a20"}
Jul 10 21:24:58.852673 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.852664    2565 scope.go:117] "RemoveContainer" containerID="3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251"
Jul 10 21:24:58.854112 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.854093    2565 generic.go:356] "Generic (PLEG): container finished" podID="18ff8b39-2fff-4781-8f00-1c4bda2c7591" containerID="5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21" exitCode=0
Jul 10 21:24:58.854233 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.854140    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"
Jul 10 21:24:58.854233 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.854145    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc" event={"ID":"18ff8b39-2fff-4781-8f00-1c4bda2c7591","Type":"ContainerDied","Data":"5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21"}
Jul 10 21:24:58.854233 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.854177    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc" event={"ID":"18ff8b39-2fff-4781-8f00-1c4bda2c7591","Type":"ContainerDied","Data":"cafb6d648d1ff1c2cbd64de02d272f3441ca26e40ead40ca6d9e3414db7626dd"}
Jul 10 21:24:58.863190 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.863175    2565 scope.go:117] "RemoveContainer" containerID="3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251"
Jul 10 21:24:58.863419 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:24:58.863401    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251\": container with ID starting with 3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251 not found: ID does not exist" containerID="3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251"
Jul 10 21:24:58.863491 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.863426    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251"} err="failed to get container status \"3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251\": rpc error: code = NotFound desc = could not find container \"3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251\": container with ID starting with 3484dcf00db473bfbfea51d71f6c25395e0c8a3485a22cbb91c76fd07ddbc251 not found: ID does not exist"
Jul 10 21:24:58.863491 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.863442    2565 scope.go:117] "RemoveContainer" containerID="5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21"
Jul 10 21:24:58.874620 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.874601    2565 scope.go:117] "RemoveContainer" containerID="5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21"
Jul 10 21:24:58.874902 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:24:58.874882    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21\": container with ID starting with 5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21 not found: ID does not exist" containerID="5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21"
Jul 10 21:24:58.874978 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.874912    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21"} err="failed to get container status \"5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21\": rpc error: code = NotFound desc = could not find container \"5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21\": container with ID starting with 5a00670c4b1a1ba9ed44930a6d479ae66a0b7b81f846bcbc2bb805ffc2c10c21 not found: ID does not exist"
Jul 10 21:24:58.879640 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.879619    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"]
Jul 10 21:24:58.883708 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.883686    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/auth-test-model-920ed7-kserve-5d9d48bd7-f884h"]
Jul 10 21:24:58.903746 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.903719    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-model-cache\") pod \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") "
Jul 10 21:24:58.903840 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.903762    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tmp-dir\") pod \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") "
Jul 10 21:24:58.903840 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.903795    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-home\") pod \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") "
Jul 10 21:24:58.903840 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.903820    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m52m\" (UniqueName: \"kubernetes.io/projected/18ff8b39-2fff-4781-8f00-1c4bda2c7591-kube-api-access-2m52m\") pod \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") "
Jul 10 21:24:58.904009 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.903856    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-dshm\") pod \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") "
Jul 10 21:24:58.904009 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.903918    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tls-certs\") pod \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\" (UID: \"18ff8b39-2fff-4781-8f00-1c4bda2c7591\") "
Jul 10 21:24:58.904009 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.903980    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-model-cache" (OuterVolumeSpecName: "model-cache") pod "18ff8b39-2fff-4781-8f00-1c4bda2c7591" (UID: "18ff8b39-2fff-4781-8f00-1c4bda2c7591"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:24:58.904156 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.904112    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-home" (OuterVolumeSpecName: "home") pod "18ff8b39-2fff-4781-8f00-1c4bda2c7591" (UID: "18ff8b39-2fff-4781-8f00-1c4bda2c7591"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:24:58.904156 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.904127    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "18ff8b39-2fff-4781-8f00-1c4bda2c7591" (UID: "18ff8b39-2fff-4781-8f00-1c4bda2c7591"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:24:58.904320 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.904301    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-home\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:58.904373 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.904324    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-model-cache\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:58.904373 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.904339    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tmp-dir\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:58.906263 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.906240    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-dshm" (OuterVolumeSpecName: "dshm") pod "18ff8b39-2fff-4781-8f00-1c4bda2c7591" (UID: "18ff8b39-2fff-4781-8f00-1c4bda2c7591"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:24:58.906594 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.906545    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18ff8b39-2fff-4781-8f00-1c4bda2c7591" (UID: "18ff8b39-2fff-4781-8f00-1c4bda2c7591"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 21:24:58.906685 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:58.906587    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18ff8b39-2fff-4781-8f00-1c4bda2c7591-kube-api-access-2m52m" (OuterVolumeSpecName: "kube-api-access-2m52m") pod "18ff8b39-2fff-4781-8f00-1c4bda2c7591" (UID: "18ff8b39-2fff-4781-8f00-1c4bda2c7591"). InnerVolumeSpecName "kube-api-access-2m52m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:24:59.004960 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:59.004916    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m52m\" (UniqueName: \"kubernetes.io/projected/18ff8b39-2fff-4781-8f00-1c4bda2c7591-kube-api-access-2m52m\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:59.004960 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:59.004940    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18ff8b39-2fff-4781-8f00-1c4bda2c7591-dshm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:59.004960 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:59.004951    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ff8b39-2fff-4781-8f00-1c4bda2c7591-tls-certs\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:24:59.181834 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:59.181800    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"]
Jul 10 21:24:59.184193 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:59.184166    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/auth-test-model-6e676e-kserve-5ff569695-48xvc"]
Jul 10 21:24:59.958518 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:59.958486    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18ff8b39-2fff-4781-8f00-1c4bda2c7591" path="/var/lib/kubelet/pods/18ff8b39-2fff-4781-8f00-1c4bda2c7591/volumes"
Jul 10 21:24:59.958984 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:24:59.958968    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" path="/var/lib/kubelet/pods/dd6a6a0e-147d-4470-8c0f-7cfd5a38a594/volumes"
Jul 10 21:25:03.478001 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:03.477953    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:25:03.478495 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:03.478030    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:25:03.490145 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:03.490122    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:25:03.896902 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:03.896870    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:25:19.150847 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.150808    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"]
Jul 10 21:25:19.151351 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.151181    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18ff8b39-2fff-4781-8f00-1c4bda2c7591" containerName="main"
Jul 10 21:25:19.151351 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.151192    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="18ff8b39-2fff-4781-8f00-1c4bda2c7591" containerName="main"
Jul 10 21:25:19.151351 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.151208    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" containerName="main"
Jul 10 21:25:19.151351 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.151213    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" containerName="main"
Jul 10 21:25:19.151351 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.151294    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="18ff8b39-2fff-4781-8f00-1c4bda2c7591" containerName="main"
Jul 10 21:25:19.151351 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.151303    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd6a6a0e-147d-4470-8c0f-7cfd5a38a594" containerName="main"
Jul 10 21:25:19.155736 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.155715    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.160071 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.160047    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"openshift-service-ca.crt\""
Jul 10 21:25:19.160255 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.160112    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"sub-test-model-6e676e-kserve-self-signed-certs\""
Jul 10 21:25:19.160255 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.160047    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"default-dockercfg-mz5st\""
Jul 10 21:25:19.160255 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.160197    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"kube-root-ca.crt\""
Jul 10 21:25:19.165048 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.165027    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"]
Jul 10 21:25:19.265926 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.265889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-home\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.266094 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.265980    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnjvz\" (UniqueName: \"kubernetes.io/projected/6a0717ea-0053-461a-8800-ddce0004e9dc-kube-api-access-bnjvz\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.266094 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.266005    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-dshm\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.266094 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.266022    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a0717ea-0053-461a-8800-ddce0004e9dc-tls-certs\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.266094 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.266081    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-tmp-dir\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.266238 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.266098    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-model-cache\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.367312 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.367276    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnjvz\" (UniqueName: \"kubernetes.io/projected/6a0717ea-0053-461a-8800-ddce0004e9dc-kube-api-access-bnjvz\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.367312 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.367315    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-dshm\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.367550 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.367333    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a0717ea-0053-461a-8800-ddce0004e9dc-tls-certs\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.367550 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.367358    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-tmp-dir\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.367550 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.367509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-model-cache\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.367751 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.367619    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-home\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.367853 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.367835    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-tmp-dir\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.367925 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.367894    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-model-cache\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.368051 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.368031    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-home\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.369757 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.369738    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-dshm\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.370008 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.369990    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a0717ea-0053-461a-8800-ddce0004e9dc-tls-certs\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.378799 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.378773    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnjvz\" (UniqueName: \"kubernetes.io/projected/6a0717ea-0053-461a-8800-ddce0004e9dc-kube-api-access-bnjvz\") pod \"sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") " pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.466920 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.466838    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:19.605570 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.605529    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"]
Jul 10 21:25:19.606218 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:25:19.606193    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a0717ea_0053_461a_8800_ddce0004e9dc.slice/crio-c1d9d06cdc11bca480428fdc5705b9612940704025dbfa004d80f20578e5a759 WatchSource:0}: Error finding container c1d9d06cdc11bca480428fdc5705b9612940704025dbfa004d80f20578e5a759: Status 404 returned error can't find the container with id c1d9d06cdc11bca480428fdc5705b9612940704025dbfa004d80f20578e5a759
Jul 10 21:25:19.982270 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.982231    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7" event={"ID":"6a0717ea-0053-461a-8800-ddce0004e9dc","Type":"ContainerStarted","Data":"9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026"}
Jul 10 21:25:19.982270 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:19.982272    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7" event={"ID":"6a0717ea-0053-461a-8800-ddce0004e9dc","Type":"ContainerStarted","Data":"c1d9d06cdc11bca480428fdc5705b9612940704025dbfa004d80f20578e5a759"}
Jul 10 21:25:20.004006 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:20.003951    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7" podStartSLOduration=1.003938163 podStartE2EDuration="1.003938163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:25:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:25:20.002943616 +0000 UTC m=+2856.495739786" watchObservedRunningTime="2026-07-10 21:25:20.003938163 +0000 UTC m=+2856.496734343"
Jul 10 21:25:29.467897 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:29.467854    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:29.468371 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:29.467915    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:29.480435 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:29.480411    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:25:30.051462 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:25:30.051435    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:26:04.835327 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:04.835292    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"]
Jul 10 21:26:05.676998 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:05.676964    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"]
Jul 10 21:26:10.664259 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.664229    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"]
Jul 10 21:26:10.667220 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.667205    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.670658 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.670639    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-920ed7\"/\"rate-test-model-920ed7-kserve-self-signed-certs\""
Jul 10 21:26:10.679257 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.679233    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"]
Jul 10 21:26:10.714127 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.714089    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txdk2\" (UniqueName: \"kubernetes.io/projected/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-kube-api-access-txdk2\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.714230 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.714154    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tls-certs\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.714230 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.714221    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-home\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.714314 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.714274    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-model-cache\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.714314 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.714307    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-dshm\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.714379 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.714337    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tmp-dir\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.815452 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.815416    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tls-certs\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.815452 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.815454    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-home\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.815682 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.815474    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-model-cache\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.815682 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.815492    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-dshm\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.815682 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.815520    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tmp-dir\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.815682 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.815632    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txdk2\" (UniqueName: \"kubernetes.io/projected/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-kube-api-access-txdk2\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.815903 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.815882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-home\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.815959 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.815935    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-model-cache\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.816038 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.816020    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tmp-dir\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.817888 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.817867    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-dshm\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.818153 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.818130    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tls-certs\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.826269 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.826245    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txdk2\" (UniqueName: \"kubernetes.io/projected/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-kube-api-access-txdk2\") pod \"rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") " pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:10.976988 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:10.976914    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:11.111941 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:11.111917    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"]
Jul 10 21:26:11.113406 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:26:11.113377    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8abead3d_fff0_40e2_a4cb_d6f2ad201fdd.slice/crio-4dad5cc43748ab8ad7455511af5369b9d3f9b7a71007f8e46fec7fd682450429 WatchSource:0}: Error finding container 4dad5cc43748ab8ad7455511af5369b9d3f9b7a71007f8e46fec7fd682450429: Status 404 returned error can't find the container with id 4dad5cc43748ab8ad7455511af5369b9d3f9b7a71007f8e46fec7fd682450429
Jul 10 21:26:11.282992 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:11.282901    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd" event={"ID":"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd","Type":"ContainerStarted","Data":"bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042"}
Jul 10 21:26:11.282992 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:11.282952    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd" event={"ID":"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd","Type":"ContainerStarted","Data":"4dad5cc43748ab8ad7455511af5369b9d3f9b7a71007f8e46fec7fd682450429"}
Jul 10 21:26:11.304227 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:11.304181    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd" podStartSLOduration=1.304166063 podStartE2EDuration="1.304166063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:26:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:26:11.302709043 +0000 UTC m=+2907.795505233" watchObservedRunningTime="2026-07-10 21:26:11.304166063 +0000 UTC m=+2907.796962242"
Jul 10 21:26:19.855995 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:19.855947    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv" podUID="05874faf-e245-4b64-a3e3-b2190421a73c" containerName="main" containerID="cri-o://5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198" gracePeriod=45
Jul 10 21:26:20.106123 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.106065    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:26:20.202600 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.202546    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-tmp-dir\") pod \"05874faf-e245-4b64-a3e3-b2190421a73c\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") "
Jul 10 21:26:20.202768 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.202622    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff5rw\" (UniqueName: \"kubernetes.io/projected/05874faf-e245-4b64-a3e3-b2190421a73c-kube-api-access-ff5rw\") pod \"05874faf-e245-4b64-a3e3-b2190421a73c\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") "
Jul 10 21:26:20.202768 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.202649    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05874faf-e245-4b64-a3e3-b2190421a73c-tls-certs\") pod \"05874faf-e245-4b64-a3e3-b2190421a73c\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") "
Jul 10 21:26:20.202898 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.202787    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-dshm\") pod \"05874faf-e245-4b64-a3e3-b2190421a73c\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") "
Jul 10 21:26:20.202898 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.202826    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-model-cache\") pod \"05874faf-e245-4b64-a3e3-b2190421a73c\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") "
Jul 10 21:26:20.202898 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.202842    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-home\") pod \"05874faf-e245-4b64-a3e3-b2190421a73c\" (UID: \"05874faf-e245-4b64-a3e3-b2190421a73c\") "
Jul 10 21:26:20.202898 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.202854    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05874faf-e245-4b64-a3e3-b2190421a73c" (UID: "05874faf-e245-4b64-a3e3-b2190421a73c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:26:20.203115 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.203063    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-model-cache" (OuterVolumeSpecName: "model-cache") pod "05874faf-e245-4b64-a3e3-b2190421a73c" (UID: "05874faf-e245-4b64-a3e3-b2190421a73c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:26:20.203173 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.203149    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-home" (OuterVolumeSpecName: "home") pod "05874faf-e245-4b64-a3e3-b2190421a73c" (UID: "05874faf-e245-4b64-a3e3-b2190421a73c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:26:20.203275 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.203258    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-tmp-dir\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:20.203335 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.203280    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-model-cache\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:20.203335 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.203292    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-home\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:20.204944 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.204921    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05874faf-e245-4b64-a3e3-b2190421a73c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05874faf-e245-4b64-a3e3-b2190421a73c" (UID: "05874faf-e245-4b64-a3e3-b2190421a73c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 21:26:20.204944 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.204928    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-dshm" (OuterVolumeSpecName: "dshm") pod "05874faf-e245-4b64-a3e3-b2190421a73c" (UID: "05874faf-e245-4b64-a3e3-b2190421a73c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:26:20.205092 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.204928    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05874faf-e245-4b64-a3e3-b2190421a73c-kube-api-access-ff5rw" (OuterVolumeSpecName: "kube-api-access-ff5rw") pod "05874faf-e245-4b64-a3e3-b2190421a73c" (UID: "05874faf-e245-4b64-a3e3-b2190421a73c"). InnerVolumeSpecName "kube-api-access-ff5rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:26:20.303945 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.303909    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05874faf-e245-4b64-a3e3-b2190421a73c-dshm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:20.303945 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.303940    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff5rw\" (UniqueName: \"kubernetes.io/projected/05874faf-e245-4b64-a3e3-b2190421a73c-kube-api-access-ff5rw\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:20.303945 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.303951    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05874faf-e245-4b64-a3e3-b2190421a73c-tls-certs\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:20.342727 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.342696    2565 generic.go:356] "Generic (PLEG): container finished" podID="05874faf-e245-4b64-a3e3-b2190421a73c" containerID="5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198" exitCode=0
Jul 10 21:26:20.342888 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.342737    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"
Jul 10 21:26:20.342888 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.342790    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv" event={"ID":"05874faf-e245-4b64-a3e3-b2190421a73c","Type":"ContainerDied","Data":"5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198"}
Jul 10 21:26:20.342888 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.342830    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv" event={"ID":"05874faf-e245-4b64-a3e3-b2190421a73c","Type":"ContainerDied","Data":"7c54754ffc2ae71ee7c5ed3dea48ea4bc74cfc8581f8ca9853d440f6788b4c19"}
Jul 10 21:26:20.342888 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.342847    2565 scope.go:117] "RemoveContainer" containerID="5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198"
Jul 10 21:26:20.352541 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.352526    2565 scope.go:117] "RemoveContainer" containerID="5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198"
Jul 10 21:26:20.352845 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:26:20.352828    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198\": container with ID starting with 5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198 not found: ID does not exist" containerID="5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198"
Jul 10 21:26:20.352904 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.352857    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198"} err="failed to get container status \"5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198\": rpc error: code = NotFound desc = could not find container \"5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198\": container with ID starting with 5e451fab9ae897cbc9d187cfc89afc7aac62fb68f4e116a49914072c8f970198 not found: ID does not exist"
Jul 10 21:26:20.366551 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.366493    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"]
Jul 10 21:26:20.370687 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.370666    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/sub-test-model-920ed7-kserve-6544778d9c-zkmsv"]
Jul 10 21:26:20.697508 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.697426    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7" podUID="6a0717ea-0053-461a-8800-ddce0004e9dc" containerName="main" containerID="cri-o://9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026" gracePeriod=45
Jul 10 21:26:20.953820 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.953760    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:26:20.977394 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.977370    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:20.977512 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.977414    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:20.994978 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:20.994956    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:21.110721 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.110695    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-home\") pod \"6a0717ea-0053-461a-8800-ddce0004e9dc\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") "
Jul 10 21:26:21.110866 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.110746    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-tmp-dir\") pod \"6a0717ea-0053-461a-8800-ddce0004e9dc\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") "
Jul 10 21:26:21.110866 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.110809    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-dshm\") pod \"6a0717ea-0053-461a-8800-ddce0004e9dc\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") "
Jul 10 21:26:21.110866 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.110837    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnjvz\" (UniqueName: \"kubernetes.io/projected/6a0717ea-0053-461a-8800-ddce0004e9dc-kube-api-access-bnjvz\") pod \"6a0717ea-0053-461a-8800-ddce0004e9dc\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") "
Jul 10 21:26:21.110866 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.110854    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a0717ea-0053-461a-8800-ddce0004e9dc-tls-certs\") pod \"6a0717ea-0053-461a-8800-ddce0004e9dc\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") "
Jul 10 21:26:21.110866 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.110869    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-model-cache\") pod \"6a0717ea-0053-461a-8800-ddce0004e9dc\" (UID: \"6a0717ea-0053-461a-8800-ddce0004e9dc\") "
Jul 10 21:26:21.111129 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.111004    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-home" (OuterVolumeSpecName: "home") pod "6a0717ea-0053-461a-8800-ddce0004e9dc" (UID: "6a0717ea-0053-461a-8800-ddce0004e9dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:26:21.111129 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.111040    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6a0717ea-0053-461a-8800-ddce0004e9dc" (UID: "6a0717ea-0053-461a-8800-ddce0004e9dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:26:21.111129 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.111120    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-home\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:21.111271 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.111137    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-tmp-dir\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:21.111271 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.111199    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "6a0717ea-0053-461a-8800-ddce0004e9dc" (UID: "6a0717ea-0053-461a-8800-ddce0004e9dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:26:21.113145 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.113124    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-dshm" (OuterVolumeSpecName: "dshm") pod "6a0717ea-0053-461a-8800-ddce0004e9dc" (UID: "6a0717ea-0053-461a-8800-ddce0004e9dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:26:21.113520 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.113496    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0717ea-0053-461a-8800-ddce0004e9dc-kube-api-access-bnjvz" (OuterVolumeSpecName: "kube-api-access-bnjvz") pod "6a0717ea-0053-461a-8800-ddce0004e9dc" (UID: "6a0717ea-0053-461a-8800-ddce0004e9dc"). InnerVolumeSpecName "kube-api-access-bnjvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:26:21.113652 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.113498    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a0717ea-0053-461a-8800-ddce0004e9dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6a0717ea-0053-461a-8800-ddce0004e9dc" (UID: "6a0717ea-0053-461a-8800-ddce0004e9dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 21:26:21.212048 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.211980    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-dshm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:21.212048 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.212007    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnjvz\" (UniqueName: \"kubernetes.io/projected/6a0717ea-0053-461a-8800-ddce0004e9dc-kube-api-access-bnjvz\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:21.212048 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.212016    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a0717ea-0053-461a-8800-ddce0004e9dc-tls-certs\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:21.212048 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.212024    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a0717ea-0053-461a-8800-ddce0004e9dc-model-cache\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:26:21.350664 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.350632    2565 generic.go:356] "Generic (PLEG): container finished" podID="6a0717ea-0053-461a-8800-ddce0004e9dc" containerID="9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026" exitCode=0
Jul 10 21:26:21.350828 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.350677    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"
Jul 10 21:26:21.350828 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.350724    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7" event={"ID":"6a0717ea-0053-461a-8800-ddce0004e9dc","Type":"ContainerDied","Data":"9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026"}
Jul 10 21:26:21.350828 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.350763    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7" event={"ID":"6a0717ea-0053-461a-8800-ddce0004e9dc","Type":"ContainerDied","Data":"c1d9d06cdc11bca480428fdc5705b9612940704025dbfa004d80f20578e5a759"}
Jul 10 21:26:21.350828 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.350780    2565 scope.go:117] "RemoveContainer" containerID="9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026"
Jul 10 21:26:21.360310 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.360293    2565 scope.go:117] "RemoveContainer" containerID="9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026"
Jul 10 21:26:21.360586 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:26:21.360542    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026\": container with ID starting with 9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026 not found: ID does not exist" containerID="9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026"
Jul 10 21:26:21.360652 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.360591    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026"} err="failed to get container status \"9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026\": rpc error: code = NotFound desc = could not find container \"9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026\": container with ID starting with 9788d609a6f2f55759a1205bd565afe6cbaa832d5581fdb73d6be2fd11063026 not found: ID does not exist"
Jul 10 21:26:21.363370 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.363348    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:26:21.372916 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.372896    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"]
Jul 10 21:26:21.376192 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.376173    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/sub-test-model-6e676e-kserve-7c45ddc79c-x9sj7"]
Jul 10 21:26:21.957736 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.957702    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05874faf-e245-4b64-a3e3-b2190421a73c" path="/var/lib/kubelet/pods/05874faf-e245-4b64-a3e3-b2190421a73c/volumes"
Jul 10 21:26:21.958104 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:21.958086    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a0717ea-0053-461a-8800-ddce0004e9dc" path="/var/lib/kubelet/pods/6a0717ea-0053-461a-8800-ddce0004e9dc/volumes"
Jul 10 21:26:37.962041 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.962009    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"]
Jul 10 21:26:37.962539 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.962524    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05874faf-e245-4b64-a3e3-b2190421a73c" containerName="main"
Jul 10 21:26:37.962607 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.962543    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="05874faf-e245-4b64-a3e3-b2190421a73c" containerName="main"
Jul 10 21:26:37.962607 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.962601    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a0717ea-0053-461a-8800-ddce0004e9dc" containerName="main"
Jul 10 21:26:37.962671 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.962610    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a0717ea-0053-461a-8800-ddce0004e9dc" containerName="main"
Jul 10 21:26:37.962715 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.962704    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a0717ea-0053-461a-8800-ddce0004e9dc" containerName="main"
Jul 10 21:26:37.962751 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.962721    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="05874faf-e245-4b64-a3e3-b2190421a73c" containerName="main"
Jul 10 21:26:37.966346 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.966326    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:37.970697 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.970672    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"openshift-service-ca.crt\""
Jul 10 21:26:37.971932 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.971909    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"default-dockercfg-mz5st\""
Jul 10 21:26:37.972046 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.971935    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"rate-test-model-6e676e-kserve-self-signed-certs\""
Jul 10 21:26:37.972046 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.971909    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6e676e\"/\"kube-root-ca.crt\""
Jul 10 21:26:37.977971 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:37.977944    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"]
Jul 10 21:26:38.158283 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.158248    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tls-certs\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.158458 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.158291    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-dshm\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.158458 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.158316    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tmp-dir\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.158458 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.158364    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-model-cache\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.158458 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.158383    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4zm9\" (UniqueName: \"kubernetes.io/projected/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-kube-api-access-x4zm9\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.158632 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.158453    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-home\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.259126 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.259044    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tls-certs\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.259126 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.259090    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-dshm\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.259126 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.259112    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tmp-dir\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.259389 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.259147    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-model-cache\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.259389 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.259181    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4zm9\" (UniqueName: \"kubernetes.io/projected/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-kube-api-access-x4zm9\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.259389 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.259235    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-home\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.259628 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.259604    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-model-cache\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.259705 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.259648    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tmp-dir\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.259705 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.259664    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-home\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.261611 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.261592    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-dshm\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.261754 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.261736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tls-certs\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.268989 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.268969    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4zm9\" (UniqueName: \"kubernetes.io/projected/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-kube-api-access-x4zm9\") pod \"rate-test-model-6e676e-kserve-649c4c7459-8rv7n\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") " pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.276898 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.276875    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:38.416405 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.416375    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"]
Jul 10 21:26:38.417798 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:26:38.417770    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b41fa5e_cc03_49eb_a43b_12cc87bfca92.slice/crio-a846b4bf968a77b3cfeeb286675f16620bc2bd16ba864500ef8d14a20cb224c1 WatchSource:0}: Error finding container a846b4bf968a77b3cfeeb286675f16620bc2bd16ba864500ef8d14a20cb224c1: Status 404 returned error can't find the container with id a846b4bf968a77b3cfeeb286675f16620bc2bd16ba864500ef8d14a20cb224c1
Jul 10 21:26:38.453993 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:38.453967    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n" event={"ID":"6b41fa5e-cc03-49eb-a43b-12cc87bfca92","Type":"ContainerStarted","Data":"a846b4bf968a77b3cfeeb286675f16620bc2bd16ba864500ef8d14a20cb224c1"}
Jul 10 21:26:39.461476 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:39.461436    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n" event={"ID":"6b41fa5e-cc03-49eb-a43b-12cc87bfca92","Type":"ContainerStarted","Data":"b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6"}
Jul 10 21:26:39.482734 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:39.482685    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n" podStartSLOduration=2.482671997 podStartE2EDuration="2.482671997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:26:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:26:39.481089725 +0000 UTC m=+2935.973885906" watchObservedRunningTime="2026-07-10 21:26:39.482671997 +0000 UTC m=+2935.975468202"
Jul 10 21:26:48.277381 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:48.277336    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:48.277877 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:48.277393    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:48.289782 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:48.289753    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:26:48.523038 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:26:48.523007    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:27:26.266579 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:26.266483    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"]
Jul 10 21:27:27.059404 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:27.059368    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"]
Jul 10 21:27:28.088386 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.088351    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7584-140217587414320"]
Jul 10 21:27:28.091876 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.091855    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217587414320"
Jul 10 21:27:28.094722 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.094700    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-psb7f\""
Jul 10 21:27:28.096455 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.096432    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7584-140217587414320"]
Jul 10 21:27:28.191196 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.191158    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h49lp\" (UniqueName: \"kubernetes.io/projected/8d704bba-18b3-4d86-a05e-39552bda543a-kube-api-access-h49lp\") pod \"test-curl-7584-140217587414320\" (UID: \"8d704bba-18b3-4d86-a05e-39552bda543a\") " pod="openshift-ingress/test-curl-7584-140217587414320"
Jul 10 21:27:28.291724 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.291682    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h49lp\" (UniqueName: \"kubernetes.io/projected/8d704bba-18b3-4d86-a05e-39552bda543a-kube-api-access-h49lp\") pod \"test-curl-7584-140217587414320\" (UID: \"8d704bba-18b3-4d86-a05e-39552bda543a\") " pod="openshift-ingress/test-curl-7584-140217587414320"
Jul 10 21:27:28.300821 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.300794    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h49lp\" (UniqueName: \"kubernetes.io/projected/8d704bba-18b3-4d86-a05e-39552bda543a-kube-api-access-h49lp\") pod \"test-curl-7584-140217587414320\" (UID: \"8d704bba-18b3-4d86-a05e-39552bda543a\") " pod="openshift-ingress/test-curl-7584-140217587414320"
Jul 10 21:27:28.403700 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.403606    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217587414320"
Jul 10 21:27:28.537630 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.537604    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7584-140217587414320"]
Jul 10 21:27:28.538118 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:27:28.538095    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d704bba_18b3_4d86_a05e_39552bda543a.slice/crio-812ed52547416a21d1830c563ca0a760b7f8ce9be095125a010f4deb0488e84b WatchSource:0}: Error finding container 812ed52547416a21d1830c563ca0a760b7f8ce9be095125a010f4deb0488e84b: Status 404 returned error can't find the container with id 812ed52547416a21d1830c563ca0a760b7f8ce9be095125a010f4deb0488e84b
Jul 10 21:27:28.539421 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.539403    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 21:27:28.737110 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:28.737022    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7584-140217587414320" event={"ID":"8d704bba-18b3-4d86-a05e-39552bda543a","Type":"ContainerStarted","Data":"812ed52547416a21d1830c563ca0a760b7f8ce9be095125a010f4deb0488e84b"}
Jul 10 21:27:29.743836 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:29.743795    2565 generic.go:356] "Generic (PLEG): container finished" podID="8d704bba-18b3-4d86-a05e-39552bda543a" containerID="99011efdedcfc69a9754b2def7d01efc24894a9a480e76131d5a52510ad12201" exitCode=0
Jul 10 21:27:29.744229 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:29.743842    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7584-140217587414320" event={"ID":"8d704bba-18b3-4d86-a05e-39552bda543a","Type":"ContainerDied","Data":"99011efdedcfc69a9754b2def7d01efc24894a9a480e76131d5a52510ad12201"}
Jul 10 21:27:30.889243 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:30.889218    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217587414320"
Jul 10 21:27:30.923521 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:30.923495    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7584-140217587414320_8d704bba-18b3-4d86-a05e-39552bda543a/test-curl-7584-140217587414320/0.log"
Jul 10 21:27:30.950807 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:30.950780    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7584-140217587414320"]
Jul 10 21:27:30.953138 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:30.953117    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7584-140217587414320"]
Jul 10 21:27:31.017483 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.017414    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h49lp\" (UniqueName: \"kubernetes.io/projected/8d704bba-18b3-4d86-a05e-39552bda543a-kube-api-access-h49lp\") pod \"8d704bba-18b3-4d86-a05e-39552bda543a\" (UID: \"8d704bba-18b3-4d86-a05e-39552bda543a\") "
Jul 10 21:27:31.019689 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.019662    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d704bba-18b3-4d86-a05e-39552bda543a-kube-api-access-h49lp" (OuterVolumeSpecName: "kube-api-access-h49lp") pod "8d704bba-18b3-4d86-a05e-39552bda543a" (UID: "8d704bba-18b3-4d86-a05e-39552bda543a"). InnerVolumeSpecName "kube-api-access-h49lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:27:31.090766 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.090731    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7584-140217587912880"]
Jul 10 21:27:31.091111 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.091098    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d704bba-18b3-4d86-a05e-39552bda543a" containerName="test-curl-7584-140217587414320"
Jul 10 21:27:31.091155 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.091113    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d704bba-18b3-4d86-a05e-39552bda543a" containerName="test-curl-7584-140217587414320"
Jul 10 21:27:31.091190 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.091182    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d704bba-18b3-4d86-a05e-39552bda543a" containerName="test-curl-7584-140217587414320"
Jul 10 21:27:31.094223 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.094202    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217587912880"
Jul 10 21:27:31.097931 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.097908    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7584-140217587912880"]
Jul 10 21:27:31.118279 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.118252    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h49lp\" (UniqueName: \"kubernetes.io/projected/8d704bba-18b3-4d86-a05e-39552bda543a-kube-api-access-h49lp\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:31.219610 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.219544    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27wrf\" (UniqueName: \"kubernetes.io/projected/7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13-kube-api-access-27wrf\") pod \"test-curl-7584-140217587912880\" (UID: \"7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13\") " pod="openshift-ingress/test-curl-7584-140217587912880"
Jul 10 21:27:31.320653 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.320613    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27wrf\" (UniqueName: \"kubernetes.io/projected/7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13-kube-api-access-27wrf\") pod \"test-curl-7584-140217587912880\" (UID: \"7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13\") " pod="openshift-ingress/test-curl-7584-140217587912880"
Jul 10 21:27:31.329290 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.329259    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27wrf\" (UniqueName: \"kubernetes.io/projected/7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13-kube-api-access-27wrf\") pod \"test-curl-7584-140217587912880\" (UID: \"7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13\") " pod="openshift-ingress/test-curl-7584-140217587912880"
Jul 10 21:27:31.404725 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.404698    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217587912880"
Jul 10 21:27:31.535768 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.535743    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7584-140217587912880"]
Jul 10 21:27:31.537362 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:27:31.537337    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7eac9f7a_e8a0_4953_84fa_35e8f7fe9e13.slice/crio-0558959509d2160abad52f63d21df8fbb0934e3fde394b268f95bd49969e1bfe WatchSource:0}: Error finding container 0558959509d2160abad52f63d21df8fbb0934e3fde394b268f95bd49969e1bfe: Status 404 returned error can't find the container with id 0558959509d2160abad52f63d21df8fbb0934e3fde394b268f95bd49969e1bfe
Jul 10 21:27:31.756202 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.756122    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217587414320"
Jul 10 21:27:31.756359 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.756121    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="812ed52547416a21d1830c563ca0a760b7f8ce9be095125a010f4deb0488e84b"
Jul 10 21:27:31.757296 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.757271    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7584-140217587912880" event={"ID":"7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13","Type":"ContainerStarted","Data":"0558959509d2160abad52f63d21df8fbb0934e3fde394b268f95bd49969e1bfe"}
Jul 10 21:27:31.766714 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:27:31.766686    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7584-140217587414320\" is forbidden: User \"system:node:ip-10-0-141-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-228.ec2.internal' and this object" podUID="8d704bba-18b3-4d86-a05e-39552bda543a" pod="openshift-ingress/test-curl-7584-140217587414320"
Jul 10 21:27:31.957935 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:31.957854    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d704bba-18b3-4d86-a05e-39552bda543a" path="/var/lib/kubelet/pods/8d704bba-18b3-4d86-a05e-39552bda543a/volumes"
Jul 10 21:27:32.764318 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:32.764280    2565 generic.go:356] "Generic (PLEG): container finished" podID="7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13" containerID="739589883deec93dca0f04e822d2d1368b9be6e1f32b773d12e438cf70be7228" exitCode=0
Jul 10 21:27:32.764493 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:32.764362    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7584-140217587912880" event={"ID":"7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13","Type":"ContainerDied","Data":"739589883deec93dca0f04e822d2d1368b9be6e1f32b773d12e438cf70be7228"}
Jul 10 21:27:33.914504 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:33.914480    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217587912880"
Jul 10 21:27:33.935003 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:33.934977    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7584-140217587912880_7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13/test-curl-7584-140217587912880/0.log"
Jul 10 21:27:33.961101 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:33.961073    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7584-140217587912880"]
Jul 10 21:27:33.965763 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:33.965738    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7584-140217587912880"]
Jul 10 21:27:34.045508 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:34.045422    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27wrf\" (UniqueName: \"kubernetes.io/projected/7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13-kube-api-access-27wrf\") pod \"7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13\" (UID: \"7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13\") "
Jul 10 21:27:34.047668 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:34.047645    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13-kube-api-access-27wrf" (OuterVolumeSpecName: "kube-api-access-27wrf") pod "7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13" (UID: "7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13"). InnerVolumeSpecName "kube-api-access-27wrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:27:34.146809 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:34.146780    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27wrf\" (UniqueName: \"kubernetes.io/projected/7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13-kube-api-access-27wrf\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:34.778010 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:34.777986    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0558959509d2160abad52f63d21df8fbb0934e3fde394b268f95bd49969e1bfe"
Jul 10 21:27:34.778177 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:34.778013    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217587912880"
Jul 10 21:27:34.788005 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:27:34.787974    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7584-140217587912880\" is forbidden: User \"system:node:ip-10-0-141-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-228.ec2.internal' and this object" podUID="7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13" pod="openshift-ingress/test-curl-7584-140217587912880"
Jul 10 21:27:35.957820 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:35.957788    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13" path="/var/lib/kubelet/pods/7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13/volumes"
Jul 10 21:27:40.082276 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.082227    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:40.082686 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.082642    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13" containerName="test-curl-7584-140217587912880"
Jul 10 21:27:40.082686 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.082654    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13" containerName="test-curl-7584-140217587912880"
Jul 10 21:27:40.082766 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.082721    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eac9f7a-e8a0-4953-84fa-35e8f7fe9e13" containerName="test-curl-7584-140217587912880"
Jul 10 21:27:40.086024 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.086007    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:40.089254 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.089225    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-psb7f\""
Jul 10 21:27:40.091180 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.091154    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:40.192058 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.192027    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kcqn\" (UniqueName: \"kubernetes.io/projected/bc2d52eb-100b-4af4-be61-5d4ab022551d-kube-api-access-2kcqn\") pod \"test-curl-7584-140217588130848\" (UID: \"bc2d52eb-100b-4af4-be61-5d4ab022551d\") " pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:40.293212 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.293155    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcqn\" (UniqueName: \"kubernetes.io/projected/bc2d52eb-100b-4af4-be61-5d4ab022551d-kube-api-access-2kcqn\") pod \"test-curl-7584-140217588130848\" (UID: \"bc2d52eb-100b-4af4-be61-5d4ab022551d\") " pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:40.302183 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.302147    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kcqn\" (UniqueName: \"kubernetes.io/projected/bc2d52eb-100b-4af4-be61-5d4ab022551d-kube-api-access-2kcqn\") pod \"test-curl-7584-140217588130848\" (UID: \"bc2d52eb-100b-4af4-be61-5d4ab022551d\") " pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:40.396406 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.396294    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:40.530773 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.530742    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:40.533984 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:27:40.533952    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc2d52eb_100b_4af4_be61_5d4ab022551d.slice/crio-150c290f787eaa31996021b9b309d6afc3af82db3f473772ac12867357d14a0c WatchSource:0}: Error finding container 150c290f787eaa31996021b9b309d6afc3af82db3f473772ac12867357d14a0c: Status 404 returned error can't find the container with id 150c290f787eaa31996021b9b309d6afc3af82db3f473772ac12867357d14a0c
Jul 10 21:27:40.815795 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:40.815762    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7584-140217588130848" event={"ID":"bc2d52eb-100b-4af4-be61-5d4ab022551d","Type":"ContainerStarted","Data":"150c290f787eaa31996021b9b309d6afc3af82db3f473772ac12867357d14a0c"}
Jul 10 21:27:41.287553 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.287457    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd" podUID="8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" containerName="main" containerID="cri-o://bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042" gracePeriod=45
Jul 10 21:27:41.351732 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.351694    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd" podUID="8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": dial tcp 10.132.0.64:8000: connect: connection refused"
Jul 10 21:27:41.547519 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.547454    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:27:41.604179 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604147    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tmp-dir\") pod \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") "
Jul 10 21:27:41.604310 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604194    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txdk2\" (UniqueName: \"kubernetes.io/projected/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-kube-api-access-txdk2\") pod \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") "
Jul 10 21:27:41.604310 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604259    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-home\") pod \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") "
Jul 10 21:27:41.604310 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604277    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-model-cache\") pod \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") "
Jul 10 21:27:41.604310 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604302    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-dshm\") pod \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") "
Jul 10 21:27:41.604479 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604343    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tls-certs\") pod \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\" (UID: \"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd\") "
Jul 10 21:27:41.604479 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604423    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" (UID: "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:27:41.604616 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604541    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-model-cache" (OuterVolumeSpecName: "model-cache") pod "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" (UID: "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:27:41.604616 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604583    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-home" (OuterVolumeSpecName: "home") pod "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" (UID: "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:27:41.604728 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604705    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-home\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:41.604728 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604727    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-model-cache\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:41.604861 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.604744    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tmp-dir\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:41.606718 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.606700    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-dshm" (OuterVolumeSpecName: "dshm") pod "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" (UID: "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:27:41.606789 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.606769    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" (UID: "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 21:27:41.606835 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.606788    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-kube-api-access-txdk2" (OuterVolumeSpecName: "kube-api-access-txdk2") pod "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" (UID: "8abead3d-fff0-40e2-a4cb-d6f2ad201fdd"). InnerVolumeSpecName "kube-api-access-txdk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:27:41.705967 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.705936    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-dshm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:41.705967 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.705961    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-tls-certs\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:41.705967 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.705971    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txdk2\" (UniqueName: \"kubernetes.io/projected/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd-kube-api-access-txdk2\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:41.823055 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.823025    2565 generic.go:356] "Generic (PLEG): container finished" podID="8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" containerID="bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042" exitCode=0
Jul 10 21:27:41.823214 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.823070    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"
Jul 10 21:27:41.823214 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.823108    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd" event={"ID":"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd","Type":"ContainerDied","Data":"bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042"}
Jul 10 21:27:41.823214 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.823154    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd" event={"ID":"8abead3d-fff0-40e2-a4cb-d6f2ad201fdd","Type":"ContainerDied","Data":"4dad5cc43748ab8ad7455511af5369b9d3f9b7a71007f8e46fec7fd682450429"}
Jul 10 21:27:41.823214 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.823175    2565 scope.go:117] "RemoveContainer" containerID="bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042"
Jul 10 21:27:41.824971 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.824944    2565 generic.go:356] "Generic (PLEG): container finished" podID="bc2d52eb-100b-4af4-be61-5d4ab022551d" containerID="04d0acd820cf2f2289808e20309489d28d9534f04e4843338e80ad5bd4a8ffa0" exitCode=0
Jul 10 21:27:41.825060 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.825010    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7584-140217588130848" event={"ID":"bc2d52eb-100b-4af4-be61-5d4ab022551d","Type":"ContainerDied","Data":"04d0acd820cf2f2289808e20309489d28d9534f04e4843338e80ad5bd4a8ffa0"}
Jul 10 21:27:41.833041 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.833023    2565 scope.go:117] "RemoveContainer" containerID="bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042"
Jul 10 21:27:41.833289 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:27:41.833270    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042\": container with ID starting with bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042 not found: ID does not exist" containerID="bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042"
Jul 10 21:27:41.833333 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.833304    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042"} err="failed to get container status \"bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042\": rpc error: code = NotFound desc = could not find container \"bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042\": container with ID starting with bcc13e6af5ee28918bf369cc53a31129b39e648a21dafdbca7ffa55cab436042 not found: ID does not exist"
Jul 10 21:27:41.854125 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.854098    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"]
Jul 10 21:27:41.857538 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.857504    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-920ed7/rate-test-model-920ed7-kserve-6cd88f4bc9-dkwxd"]
Jul 10 21:27:41.958402 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:41.958368    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" path="/var/lib/kubelet/pods/8abead3d-fff0-40e2-a4cb-d6f2ad201fdd/volumes"
Jul 10 21:27:42.079772 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.079674    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n" podUID="6b41fa5e-cc03-49eb-a43b-12cc87bfca92" containerName="main" containerID="cri-o://b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6" gracePeriod=45
Jul 10 21:27:42.320603 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.320580    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:27:42.413176 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413145    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-dshm\") pod \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") "
Jul 10 21:27:42.413324 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413191    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-home\") pod \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") "
Jul 10 21:27:42.413324 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413229    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tmp-dir\") pod \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") "
Jul 10 21:27:42.413324 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413273    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4zm9\" (UniqueName: \"kubernetes.io/projected/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-kube-api-access-x4zm9\") pod \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") "
Jul 10 21:27:42.413511 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413353    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-model-cache\") pod \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") "
Jul 10 21:27:42.413511 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413383    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tls-certs\") pod \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\" (UID: \"6b41fa5e-cc03-49eb-a43b-12cc87bfca92\") "
Jul 10 21:27:42.413511 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413497    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b41fa5e-cc03-49eb-a43b-12cc87bfca92" (UID: "6b41fa5e-cc03-49eb-a43b-12cc87bfca92"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:27:42.413694 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413507    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-home" (OuterVolumeSpecName: "home") pod "6b41fa5e-cc03-49eb-a43b-12cc87bfca92" (UID: "6b41fa5e-cc03-49eb-a43b-12cc87bfca92"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:27:42.413694 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413650    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b41fa5e-cc03-49eb-a43b-12cc87bfca92" (UID: "6b41fa5e-cc03-49eb-a43b-12cc87bfca92"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:27:42.413806 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413734    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-home\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:42.413806 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413749    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tmp-dir\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:42.413806 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.413759    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-model-cache\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:42.415439 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.415419    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-dshm" (OuterVolumeSpecName: "dshm") pod "6b41fa5e-cc03-49eb-a43b-12cc87bfca92" (UID: "6b41fa5e-cc03-49eb-a43b-12cc87bfca92"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 21:27:42.415492 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.415454    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-kube-api-access-x4zm9" (OuterVolumeSpecName: "kube-api-access-x4zm9") pod "6b41fa5e-cc03-49eb-a43b-12cc87bfca92" (UID: "6b41fa5e-cc03-49eb-a43b-12cc87bfca92"). InnerVolumeSpecName "kube-api-access-x4zm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:27:42.415602 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.415583    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b41fa5e-cc03-49eb-a43b-12cc87bfca92" (UID: "6b41fa5e-cc03-49eb-a43b-12cc87bfca92"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 21:27:42.514533 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.514501    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-dshm\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:42.514533 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.514527    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4zm9\" (UniqueName: \"kubernetes.io/projected/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-kube-api-access-x4zm9\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:42.514533 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.514536    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b41fa5e-cc03-49eb-a43b-12cc87bfca92-tls-certs\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:42.839429 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.839396    2565 generic.go:356] "Generic (PLEG): container finished" podID="6b41fa5e-cc03-49eb-a43b-12cc87bfca92" containerID="b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6" exitCode=0
Jul 10 21:27:42.839662 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.839448    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"
Jul 10 21:27:42.839662 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.839484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n" event={"ID":"6b41fa5e-cc03-49eb-a43b-12cc87bfca92","Type":"ContainerDied","Data":"b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6"}
Jul 10 21:27:42.839662 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.839532    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n" event={"ID":"6b41fa5e-cc03-49eb-a43b-12cc87bfca92","Type":"ContainerDied","Data":"a846b4bf968a77b3cfeeb286675f16620bc2bd16ba864500ef8d14a20cb224c1"}
Jul 10 21:27:42.839662 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.839553    2565 scope.go:117] "RemoveContainer" containerID="b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6"
Jul 10 21:27:42.849997 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.849976    2565 scope.go:117] "RemoveContainer" containerID="b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6"
Jul 10 21:27:42.850282 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:27:42.850262    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6\": container with ID starting with b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6 not found: ID does not exist" containerID="b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6"
Jul 10 21:27:42.850384 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.850290    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6"} err="failed to get container status \"b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6\": rpc error: code = NotFound desc = could not find container \"b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6\": container with ID starting with b659cc5700d5a1676f1ead45794041472128561deca0247eb9e387a5cd3031d6 not found: ID does not exist"
Jul 10 21:27:42.861926 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.861902    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"]
Jul 10 21:27:42.865482 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.865461    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6e676e/rate-test-model-6e676e-kserve-649c4c7459-8rv7n"]
Jul 10 21:27:42.980059 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:42.980037    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:43.000732 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.000706    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7584-140217588130848_bc2d52eb-100b-4af4-be61-5d4ab022551d/test-curl-7584-140217588130848/0.log"
Jul 10 21:27:43.019207 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.019180    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kcqn\" (UniqueName: \"kubernetes.io/projected/bc2d52eb-100b-4af4-be61-5d4ab022551d-kube-api-access-2kcqn\") pod \"bc2d52eb-100b-4af4-be61-5d4ab022551d\" (UID: \"bc2d52eb-100b-4af4-be61-5d4ab022551d\") "
Jul 10 21:27:43.021281 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.021259    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc2d52eb-100b-4af4-be61-5d4ab022551d-kube-api-access-2kcqn" (OuterVolumeSpecName: "kube-api-access-2kcqn") pod "bc2d52eb-100b-4af4-be61-5d4ab022551d" (UID: "bc2d52eb-100b-4af4-be61-5d4ab022551d"). InnerVolumeSpecName "kube-api-access-2kcqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:27:43.028018 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.027994    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:43.031495 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.031462    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:43.120135 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.120053    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kcqn\" (UniqueName: \"kubernetes.io/projected/bc2d52eb-100b-4af4-be61-5d4ab022551d-kube-api-access-2kcqn\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:43.157619 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.157573    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:43.157987 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.157974    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc2d52eb-100b-4af4-be61-5d4ab022551d" containerName="test-curl-7584-140217588130848"
Jul 10 21:27:43.158043 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.157989    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc2d52eb-100b-4af4-be61-5d4ab022551d" containerName="test-curl-7584-140217588130848"
Jul 10 21:27:43.158043 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.158004    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b41fa5e-cc03-49eb-a43b-12cc87bfca92" containerName="main"
Jul 10 21:27:43.158043 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.158009    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b41fa5e-cc03-49eb-a43b-12cc87bfca92" containerName="main"
Jul 10 21:27:43.158043 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.158022    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" containerName="main"
Jul 10 21:27:43.158043 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.158026    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" containerName="main"
Jul 10 21:27:43.158195 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.158083    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b41fa5e-cc03-49eb-a43b-12cc87bfca92" containerName="main"
Jul 10 21:27:43.158195 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.158093    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8abead3d-fff0-40e2-a4cb-d6f2ad201fdd" containerName="main"
Jul 10 21:27:43.158195 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.158099    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc2d52eb-100b-4af4-be61-5d4ab022551d" containerName="test-curl-7584-140217588130848"
Jul 10 21:27:43.161162 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.161144    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:43.165483 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.165459    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:43.221373 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.221344    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmlxt\" (UniqueName: \"kubernetes.io/projected/2524e5ca-0b2a-4dd5-a3db-27c800136e0d-kube-api-access-gmlxt\") pod \"test-curl-7584-140217588130848\" (UID: \"2524e5ca-0b2a-4dd5-a3db-27c800136e0d\") " pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:43.321997 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.321955    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmlxt\" (UniqueName: \"kubernetes.io/projected/2524e5ca-0b2a-4dd5-a3db-27c800136e0d-kube-api-access-gmlxt\") pod \"test-curl-7584-140217588130848\" (UID: \"2524e5ca-0b2a-4dd5-a3db-27c800136e0d\") " pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:43.330606 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.330581    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmlxt\" (UniqueName: \"kubernetes.io/projected/2524e5ca-0b2a-4dd5-a3db-27c800136e0d-kube-api-access-gmlxt\") pod \"test-curl-7584-140217588130848\" (UID: \"2524e5ca-0b2a-4dd5-a3db-27c800136e0d\") " pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:43.471866 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.471786    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:43.620760 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.620724    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:43.622499 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:27:43.622473    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2524e5ca_0b2a_4dd5_a3db_27c800136e0d.slice/crio-4751f42447967e4a2f6039d57937668f49daab769dd3596a1c7936c4a28dcaa5 WatchSource:0}: Error finding container 4751f42447967e4a2f6039d57937668f49daab769dd3596a1c7936c4a28dcaa5: Status 404 returned error can't find the container with id 4751f42447967e4a2f6039d57937668f49daab769dd3596a1c7936c4a28dcaa5
Jul 10 21:27:43.850991 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.850962    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:43.850991 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.850987    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="150c290f787eaa31996021b9b309d6afc3af82db3f473772ac12867357d14a0c"
Jul 10 21:27:43.852219 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.852191    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7584-140217588130848" event={"ID":"2524e5ca-0b2a-4dd5-a3db-27c800136e0d","Type":"ContainerStarted","Data":"4751f42447967e4a2f6039d57937668f49daab769dd3596a1c7936c4a28dcaa5"}
Jul 10 21:27:43.862236 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.862211    2565 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7584-140217588130848" oldPodUID="bc2d52eb-100b-4af4-be61-5d4ab022551d" podUID="2524e5ca-0b2a-4dd5-a3db-27c800136e0d"
Jul 10 21:27:43.959144 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.959115    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b41fa5e-cc03-49eb-a43b-12cc87bfca92" path="/var/lib/kubelet/pods/6b41fa5e-cc03-49eb-a43b-12cc87bfca92/volumes"
Jul 10 21:27:43.959490 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:43.959477    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc2d52eb-100b-4af4-be61-5d4ab022551d" path="/var/lib/kubelet/pods/bc2d52eb-100b-4af4-be61-5d4ab022551d/volumes"
Jul 10 21:27:44.462322 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:44.462212    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:27:44.487342 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:44.468003    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:27:44.487342 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:44.469836    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:27:44.487342 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:44.475197    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ndzlv_00c5533a-15c8-4090-9d1a-e5e6ae42ff28/ovn-acl-logging/0.log"
Jul 10 21:27:44.858922 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:44.858886    2565 generic.go:356] "Generic (PLEG): container finished" podID="2524e5ca-0b2a-4dd5-a3db-27c800136e0d" containerID="1a354b285abb7a68e8942fa13e545d9d87c2dd697767cafd6606d6c6ffa8ae64" exitCode=0
Jul 10 21:27:44.859095 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:44.858974    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7584-140217588130848" event={"ID":"2524e5ca-0b2a-4dd5-a3db-27c800136e0d","Type":"ContainerDied","Data":"1a354b285abb7a68e8942fa13e545d9d87c2dd697767cafd6606d6c6ffa8ae64"}
Jul 10 21:27:46.007110 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.007086    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:46.027821 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.027793    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7584-140217588130848_2524e5ca-0b2a-4dd5-a3db-27c800136e0d/test-curl-7584-140217588130848/0.log"
Jul 10 21:27:46.044041 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.044016    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmlxt\" (UniqueName: \"kubernetes.io/projected/2524e5ca-0b2a-4dd5-a3db-27c800136e0d-kube-api-access-gmlxt\") pod \"2524e5ca-0b2a-4dd5-a3db-27c800136e0d\" (UID: \"2524e5ca-0b2a-4dd5-a3db-27c800136e0d\") "
Jul 10 21:27:46.046123 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.046094    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2524e5ca-0b2a-4dd5-a3db-27c800136e0d-kube-api-access-gmlxt" (OuterVolumeSpecName: "kube-api-access-gmlxt") pod "2524e5ca-0b2a-4dd5-a3db-27c800136e0d" (UID: "2524e5ca-0b2a-4dd5-a3db-27c800136e0d"). InnerVolumeSpecName "kube-api-access-gmlxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 21:27:46.053649 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.053626    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:46.056976 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.056953    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7584-140217588130848"]
Jul 10 21:27:46.145012 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.144940    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmlxt\" (UniqueName: \"kubernetes.io/projected/2524e5ca-0b2a-4dd5-a3db-27c800136e0d-kube-api-access-gmlxt\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:46.253666 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.253638    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"]
Jul 10 21:27:46.254038 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.254025    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2524e5ca-0b2a-4dd5-a3db-27c800136e0d" containerName="test-curl-7584-140217588130848"
Jul 10 21:27:46.254083 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.254040    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2524e5ca-0b2a-4dd5-a3db-27c800136e0d" containerName="test-curl-7584-140217588130848"
Jul 10 21:27:46.254118 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.254108    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2524e5ca-0b2a-4dd5-a3db-27c800136e0d" containerName="test-curl-7584-140217588130848"
Jul 10 21:27:46.257388 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.257372    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"
Jul 10 21:27:46.260342 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.260148    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 21:27:46.260470 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.260403    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:27:46.263959 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.263934    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"]
Jul 10 21:27:46.347132 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.347097    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e68a37-72f0-4bea-8351-dcf4b31989c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9\" (UID: \"d9e68a37-72f0-4bea-8351-dcf4b31989c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"
Jul 10 21:27:46.448473 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.448371    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e68a37-72f0-4bea-8351-dcf4b31989c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9\" (UID: \"d9e68a37-72f0-4bea-8351-dcf4b31989c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"
Jul 10 21:27:46.449073 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.449049    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e68a37-72f0-4bea-8351-dcf4b31989c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9\" (UID: \"d9e68a37-72f0-4bea-8351-dcf4b31989c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"
Jul 10 21:27:46.570278 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.570238    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"
Jul 10 21:27:46.704419 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.704340    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"]
Jul 10 21:27:46.708680 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:27:46.708652    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e68a37_72f0_4bea_8351_dcf4b31989c9.slice/crio-cfc5c806121503f98929bbec07d333365d631e5fbf2c12df4351ff881d2fd922 WatchSource:0}: Error finding container cfc5c806121503f98929bbec07d333365d631e5fbf2c12df4351ff881d2fd922: Status 404 returned error can't find the container with id cfc5c806121503f98929bbec07d333365d631e5fbf2c12df4351ff881d2fd922
Jul 10 21:27:46.871891 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.871866    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:46.872070 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.871904    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4751f42447967e4a2f6039d57937668f49daab769dd3596a1c7936c4a28dcaa5"
Jul 10 21:27:46.873289 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:46.873251    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9" event={"ID":"d9e68a37-72f0-4bea-8351-dcf4b31989c9","Type":"ContainerStarted","Data":"cfc5c806121503f98929bbec07d333365d631e5fbf2c12df4351ff881d2fd922"}
Jul 10 21:27:46.882049 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:27:46.882022    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7584-140217588130848\" is forbidden: User \"system:node:ip-10-0-141-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-228.ec2.internal' and this object" podUID="2524e5ca-0b2a-4dd5-a3db-27c800136e0d" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:47.880204 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:47.880170    2565 generic.go:356] "Generic (PLEG): container finished" podID="d9e68a37-72f0-4bea-8351-dcf4b31989c9" containerID="293a27ad04e390a088ff139fd414ba876d8febf4b808528b812e4af0df899213" exitCode=0
Jul 10 21:27:47.880604 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:47.880245    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9" event={"ID":"d9e68a37-72f0-4bea-8351-dcf4b31989c9","Type":"ContainerDied","Data":"293a27ad04e390a088ff139fd414ba876d8febf4b808528b812e4af0df899213"}
Jul 10 21:27:47.897621 ip-10-0-141-228 kubenswrapper[2565]: E0710 21:27:47.897584    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7584-140217588130848\" is forbidden: User \"system:node:ip-10-0-141-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-228.ec2.internal' and this object" podUID="2524e5ca-0b2a-4dd5-a3db-27c800136e0d" pod="openshift-ingress/test-curl-7584-140217588130848"
Jul 10 21:27:47.957904 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:47.957871    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2524e5ca-0b2a-4dd5-a3db-27c800136e0d" path="/var/lib/kubelet/pods/2524e5ca-0b2a-4dd5-a3db-27c800136e0d/volumes"
Jul 10 21:27:49.021961 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:49.021933    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"
Jul 10 21:27:49.073619 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:49.073588    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e68a37-72f0-4bea-8351-dcf4b31989c9-maas-api-service-ca\") pod \"d9e68a37-72f0-4bea-8351-dcf4b31989c9\" (UID: \"d9e68a37-72f0-4bea-8351-dcf4b31989c9\") "
Jul 10 21:27:49.078590 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:49.074249    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9e68a37-72f0-4bea-8351-dcf4b31989c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9e68a37-72f0-4bea-8351-dcf4b31989c9" (UID: "d9e68a37-72f0-4bea-8351-dcf4b31989c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 21:27:49.078590 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:49.074491    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9e68a37-72f0-4bea-8351-dcf4b31989c9-maas-api-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:27:49.893192 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:49.893161    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9"
Jul 10 21:27:49.893366 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:49.893194    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9" event={"ID":"d9e68a37-72f0-4bea-8351-dcf4b31989c9","Type":"ContainerDied","Data":"cfc5c806121503f98929bbec07d333365d631e5fbf2c12df4351ff881d2fd922"}
Jul 10 21:27:49.893366 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:27:49.893269    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfc5c806121503f98929bbec07d333365d631e5fbf2c12df4351ff881d2fd922"
Jul 10 21:28:12.426634 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.426596    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"]
Jul 10 21:28:12.427138 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.427011    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9e68a37-72f0-4bea-8351-dcf4b31989c9" containerName="revoke-keys"
Jul 10 21:28:12.427138 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.427022    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9e68a37-72f0-4bea-8351-dcf4b31989c9" containerName="revoke-keys"
Jul 10 21:28:12.427138 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.427087    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9e68a37-72f0-4bea-8351-dcf4b31989c9" containerName="revoke-keys"
Jul 10 21:28:12.432373 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.432357    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"
Jul 10 21:28:12.436230 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.436095    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\""
Jul 10 21:28:12.436230 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.436117    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 21:28:12.438378 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.438358    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"]
Jul 10 21:28:12.585625 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.585589    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668e9ca1-3421-4024-8dc9-ac35928ade7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62\" (UID: \"668e9ca1-3421-4024-8dc9-ac35928ade7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"
Jul 10 21:28:12.686619 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.686509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668e9ca1-3421-4024-8dc9-ac35928ade7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62\" (UID: \"668e9ca1-3421-4024-8dc9-ac35928ade7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"
Jul 10 21:28:12.687139 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.687118    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668e9ca1-3421-4024-8dc9-ac35928ade7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62\" (UID: \"668e9ca1-3421-4024-8dc9-ac35928ade7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"
Jul 10 21:28:12.744651 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.744609    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"
Jul 10 21:28:12.884239 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:12.884213    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"]
Jul 10 21:28:12.885887 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:28:12.885859    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod668e9ca1_3421_4024_8dc9_ac35928ade7b.slice/crio-e3bfe59731ee6e9c874addb6d1accd094a0e13d0165a03ed467b23806c55850d WatchSource:0}: Error finding container e3bfe59731ee6e9c874addb6d1accd094a0e13d0165a03ed467b23806c55850d: Status 404 returned error can't find the container with id e3bfe59731ee6e9c874addb6d1accd094a0e13d0165a03ed467b23806c55850d
Jul 10 21:28:13.037620 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:13.037505    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62" event={"ID":"668e9ca1-3421-4024-8dc9-ac35928ade7b","Type":"ContainerStarted","Data":"e3bfe59731ee6e9c874addb6d1accd094a0e13d0165a03ed467b23806c55850d"}
Jul 10 21:28:14.044764 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:14.044672    2565 generic.go:356] "Generic (PLEG): container finished" podID="668e9ca1-3421-4024-8dc9-ac35928ade7b" containerID="27f0050b32416f978ea57008460e8d852e3aa4d19e9f98df11adf6a5edcbc6eb" exitCode=0
Jul 10 21:28:14.045120 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:14.044759    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62" event={"ID":"668e9ca1-3421-4024-8dc9-ac35928ade7b","Type":"ContainerDied","Data":"27f0050b32416f978ea57008460e8d852e3aa4d19e9f98df11adf6a5edcbc6eb"}
Jul 10 21:28:15.186321 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:15.186297    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"
Jul 10 21:28:15.312431 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:15.312330    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668e9ca1-3421-4024-8dc9-ac35928ade7b-maas-api-service-ca\") pod \"668e9ca1-3421-4024-8dc9-ac35928ade7b\" (UID: \"668e9ca1-3421-4024-8dc9-ac35928ade7b\") "
Jul 10 21:28:15.312796 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:15.312764    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/668e9ca1-3421-4024-8dc9-ac35928ade7b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "668e9ca1-3421-4024-8dc9-ac35928ade7b" (UID: "668e9ca1-3421-4024-8dc9-ac35928ade7b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 21:28:15.413692 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:15.413658    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668e9ca1-3421-4024-8dc9-ac35928ade7b-maas-api-service-ca\") on node \"ip-10-0-141-228.ec2.internal\" DevicePath \"\""
Jul 10 21:28:16.058596 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:16.058536    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62" event={"ID":"668e9ca1-3421-4024-8dc9-ac35928ade7b","Type":"ContainerDied","Data":"e3bfe59731ee6e9c874addb6d1accd094a0e13d0165a03ed467b23806c55850d"}
Jul 10 21:28:16.058762 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:16.058605    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62"
Jul 10 21:28:16.058762 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:16.058610    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3bfe59731ee6e9c874addb6d1accd094a0e13d0165a03ed467b23806c55850d"
Jul 10 21:28:38.882035 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:38.881955    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-66bfb49bfc-q5j4g_78c17974-b45b-442a-af5a-b9f9bba6773a/authorino/0.log"
Jul 10 21:28:44.407777 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:44.407747    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6558bffdbc-b2zgh_cc7d9625-4d68-4299-8ad0-82d8779118d7/copy-manifests/0.log"
Jul 10 21:28:44.414127 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:44.414103    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6558bffdbc-b2zgh_cc7d9625-4d68-4299-8ad0-82d8779118d7/manager/0.log"
Jul 10 21:28:45.307129 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:45.307101    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29728635-gtkmn_223c86e1-dc55-4c2d-93f2-307078dd0322/cleanup/0.log"
Jul 10 21:28:45.652369 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:45.652291    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-ed1347-9p9vw_5e11a7e4-9e8e-48c2-b9bb-54d4593a3c3f/revoke-keys/0.log"
Jul 10 21:28:45.766686 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:45.766656    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-c4ebda-mkgzq_e416369d-513e-4c1c-aed5-b71ad1b36d52/revoke-keys/0.log"
Jul 10 21:28:46.099409 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:46.099373    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-5f5b2e-btfbp_aec122bc-4030-4deb-a098-6ab8aaec43d3/revoke-keys/0.log"
Jul 10 21:28:46.212227 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:46.212194    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-6fde404a-822k4_ea09632b-40c6-4f53-871b-2d5d7b6b6b07/revoke-keys/0.log"
Jul 10 21:28:46.428082 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:46.428010    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-dd381648-psl2w_d6454bc2-2276-4d0b-acad-462ef4284c6f/revoke-keys/0.log"
Jul 10 21:28:46.540037 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:46.539983    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-920ed7-cc2s9_d9e68a37-72f0-4bea-8351-dcf4b31989c9/revoke-keys/0.log"
Jul 10 21:28:46.651785 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:46.651757    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-6e676e-lgt62_668e9ca1-3421-4024-8dc9-ac35928ade7b/revoke-keys/0.log"
Jul 10 21:28:47.537500 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.537469    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96_1528f9d5-c452-4658-ab5d-defe6aebe5b6/util/0.log"
Jul 10 21:28:47.543143 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.543124    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96_1528f9d5-c452-4658-ab5d-defe6aebe5b6/pull/0.log"
Jul 10 21:28:47.548490 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.548471    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96_1528f9d5-c452-4658-ab5d-defe6aebe5b6/extract/0.log"
Jul 10 21:28:47.656527 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.656505    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5_c8205fd1-e2ec-418c-9011-ba1acfbed1fa/util/0.log"
Jul 10 21:28:47.662241 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.662222    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5_c8205fd1-e2ec-418c-9011-ba1acfbed1fa/pull/0.log"
Jul 10 21:28:47.667752 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.667730    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5_c8205fd1-e2ec-418c-9011-ba1acfbed1fa/extract/0.log"
Jul 10 21:28:47.778512 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.778485    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd_5112aa1a-f16f-49c5-875a-3653fd6a3a45/util/0.log"
Jul 10 21:28:47.784706 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.784685    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd_5112aa1a-f16f-49c5-875a-3653fd6a3a45/pull/0.log"
Jul 10 21:28:47.790798 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.790731    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd_5112aa1a-f16f-49c5-875a-3653fd6a3a45/extract/0.log"
Jul 10 21:28:47.900237 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.900209    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8_2d6eda7a-3957-49dd-baff-d74160f88abd/util/0.log"
Jul 10 21:28:47.906267 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.906244    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8_2d6eda7a-3957-49dd-baff-d74160f88abd/pull/0.log"
Jul 10 21:28:47.912372 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:47.912352    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8_2d6eda7a-3957-49dd-baff-d74160f88abd/extract/0.log"
Jul 10 21:28:48.034087 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:48.034048    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-66bfb49bfc-q5j4g_78c17974-b45b-442a-af5a-b9f9bba6773a/authorino/0.log"
Jul 10 21:28:48.877972 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:48.877927    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wnsj8_09abe612-6421-49d1-be23-6b9fefea04e4/manager/0.log"
Jul 10 21:28:49.464250 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:49.464220    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86d9dbc65b-5wdn8_c98ed95a-26fc-4dc8-880b-5c379f0d19b6/kube-auth-proxy/0.log"
Jul 10 21:28:50.650307 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:50.650281    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jtq7t_9cd84d43-c27e-4f09-befd-928d7c99dd45/main/0.log"
Jul 10 21:28:50.986408 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:50.986336    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-k567x_03765a0f-c646-4c3d-81d2-abd9d4f75937/main/0.log"
Jul 10 21:28:55.147047 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.147012    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bhd5q/must-gather-lb22b"]
Jul 10 21:28:55.147495 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.147385    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="668e9ca1-3421-4024-8dc9-ac35928ade7b" containerName="revoke-keys"
Jul 10 21:28:55.147495 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.147396    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="668e9ca1-3421-4024-8dc9-ac35928ade7b" containerName="revoke-keys"
Jul 10 21:28:55.147495 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.147472    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="668e9ca1-3421-4024-8dc9-ac35928ade7b" containerName="revoke-keys"
Jul 10 21:28:55.150675 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.150657    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bhd5q/must-gather-lb22b"
Jul 10 21:28:55.153350 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.153327    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bhd5q\"/\"openshift-service-ca.crt\""
Jul 10 21:28:55.153482 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.153329    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bhd5q\"/\"kube-root-ca.crt\""
Jul 10 21:28:55.154437 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.154412    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bhd5q\"/\"default-dockercfg-blt7c\""
Jul 10 21:28:55.157709 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.157684    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bhd5q/must-gather-lb22b"]
Jul 10 21:28:55.261333 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.261298    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jsvv\" (UniqueName: \"kubernetes.io/projected/3a6e2301-bfd0-4b20-b206-e828a7a87d69-kube-api-access-8jsvv\") pod \"must-gather-lb22b\" (UID: \"3a6e2301-bfd0-4b20-b206-e828a7a87d69\") " pod="openshift-must-gather-bhd5q/must-gather-lb22b"
Jul 10 21:28:55.261509 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.261350    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3a6e2301-bfd0-4b20-b206-e828a7a87d69-must-gather-output\") pod \"must-gather-lb22b\" (UID: \"3a6e2301-bfd0-4b20-b206-e828a7a87d69\") " pod="openshift-must-gather-bhd5q/must-gather-lb22b"
Jul 10 21:28:55.362720 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.362688    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jsvv\" (UniqueName: \"kubernetes.io/projected/3a6e2301-bfd0-4b20-b206-e828a7a87d69-kube-api-access-8jsvv\") pod \"must-gather-lb22b\" (UID: \"3a6e2301-bfd0-4b20-b206-e828a7a87d69\") " pod="openshift-must-gather-bhd5q/must-gather-lb22b"
Jul 10 21:28:55.362914 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.362730    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3a6e2301-bfd0-4b20-b206-e828a7a87d69-must-gather-output\") pod \"must-gather-lb22b\" (UID: \"3a6e2301-bfd0-4b20-b206-e828a7a87d69\") " pod="openshift-must-gather-bhd5q/must-gather-lb22b"
Jul 10 21:28:55.363053 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.363037    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3a6e2301-bfd0-4b20-b206-e828a7a87d69-must-gather-output\") pod \"must-gather-lb22b\" (UID: \"3a6e2301-bfd0-4b20-b206-e828a7a87d69\") " pod="openshift-must-gather-bhd5q/must-gather-lb22b"
Jul 10 21:28:55.371383 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.371361    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jsvv\" (UniqueName: \"kubernetes.io/projected/3a6e2301-bfd0-4b20-b206-e828a7a87d69-kube-api-access-8jsvv\") pod \"must-gather-lb22b\" (UID: \"3a6e2301-bfd0-4b20-b206-e828a7a87d69\") " pod="openshift-must-gather-bhd5q/must-gather-lb22b"
Jul 10 21:28:55.461017 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.460934    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bhd5q/must-gather-lb22b"
Jul 10 21:28:55.599930 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:55.599904    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bhd5q/must-gather-lb22b"]
Jul 10 21:28:55.601583 ip-10-0-141-228 kubenswrapper[2565]: W0710 21:28:55.601531    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a6e2301_bfd0_4b20_b206_e828a7a87d69.slice/crio-08390b85471ae54a3b31b9948ccbf26ceeb284acd90efb1d5ba1f5e28523c6ba WatchSource:0}: Error finding container 08390b85471ae54a3b31b9948ccbf26ceeb284acd90efb1d5ba1f5e28523c6ba: Status 404 returned error can't find the container with id 08390b85471ae54a3b31b9948ccbf26ceeb284acd90efb1d5ba1f5e28523c6ba
Jul 10 21:28:56.292328 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:56.292289    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bhd5q/must-gather-lb22b" event={"ID":"3a6e2301-bfd0-4b20-b206-e828a7a87d69","Type":"ContainerStarted","Data":"08390b85471ae54a3b31b9948ccbf26ceeb284acd90efb1d5ba1f5e28523c6ba"}
Jul 10 21:28:57.303486 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:57.302605    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bhd5q/must-gather-lb22b" event={"ID":"3a6e2301-bfd0-4b20-b206-e828a7a87d69","Type":"ContainerStarted","Data":"82646c198ec9a7f723436526faf85a202e45e01f159effae9ab45e9068da6a0e"}
Jul 10 21:28:57.303486 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:57.302663    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bhd5q/must-gather-lb22b" event={"ID":"3a6e2301-bfd0-4b20-b206-e828a7a87d69","Type":"ContainerStarted","Data":"ebf10d7f193565f7bb81108f67fb7c0e4e9c615aa709aa3a3a545befdcaf5030"}
Jul 10 21:28:57.322898 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:57.322831    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bhd5q/must-gather-lb22b" podStartSLOduration=1.434221903 podStartE2EDuration="2.322811382s" totalImagesPullingTime="888.589479ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:28:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:28:57.318941839 +0000 UTC m=+3073.811738020" watchObservedRunningTime="2026-07-10 21:28:57.322811382 +0000 UTC m=+3073.815607563"
Jul 10 21:28:57.978725 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:57.978695    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pdnrc_80519b7a-2d8b-4103-9b9d-df134ba3845e/global-pull-secret-syncer/0.log"
Jul 10 21:28:58.149239 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:58.149207    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-svbpz_44d5f133-ec15-48c7-8fb5-26dbf7ce9106/konnectivity-agent/0.log"
Jul 10 21:28:58.192162 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:28:58.192118    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-228.ec2.internal_b3a5470e748fa4893099e28e4ea31d34/haproxy/0.log"
Jul 10 21:29:02.642807 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.642765    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96_1528f9d5-c452-4658-ab5d-defe6aebe5b6/extract/0.log"
Jul 10 21:29:02.666516 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.666436    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96_1528f9d5-c452-4658-ab5d-defe6aebe5b6/util/0.log"
Jul 10 21:29:02.690083 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.690030    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759g9m96_1528f9d5-c452-4658-ab5d-defe6aebe5b6/pull/0.log"
Jul 10 21:29:02.714452 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.714418    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5_c8205fd1-e2ec-418c-9011-ba1acfbed1fa/extract/0.log"
Jul 10 21:29:02.736088 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.736063    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5_c8205fd1-e2ec-418c-9011-ba1acfbed1fa/util/0.log"
Jul 10 21:29:02.756329 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.756304    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k5cs5_c8205fd1-e2ec-418c-9011-ba1acfbed1fa/pull/0.log"
Jul 10 21:29:02.787776 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.787711    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd_5112aa1a-f16f-49c5-875a-3653fd6a3a45/extract/0.log"
Jul 10 21:29:02.807719 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.807649    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd_5112aa1a-f16f-49c5-875a-3653fd6a3a45/util/0.log"
Jul 10 21:29:02.833450 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.833388    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hctcd_5112aa1a-f16f-49c5-875a-3653fd6a3a45/pull/0.log"
Jul 10 21:29:02.864416 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.864344    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8_2d6eda7a-3957-49dd-baff-d74160f88abd/extract/0.log"
Jul 10 21:29:02.884190 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.884166    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8_2d6eda7a-3957-49dd-baff-d74160f88abd/util/0.log"
Jul 10 21:29:02.903845 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:02.903764    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1njpg8_2d6eda7a-3957-49dd-baff-d74160f88abd/pull/0.log"
Jul 10 21:29:03.299642 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:03.299531    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-66bfb49bfc-q5j4g_78c17974-b45b-442a-af5a-b9f9bba6773a/authorino/0.log"
Jul 10 21:29:03.720294 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:03.720261    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wnsj8_09abe612-6421-49d1-be23-6b9fefea04e4/manager/0.log"
Jul 10 21:29:05.492950 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.492914    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4/alertmanager/0.log"
Jul 10 21:29:05.513684 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.513600    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4/config-reloader/0.log"
Jul 10 21:29:05.533503 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.533466    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4/kube-rbac-proxy-web/0.log"
Jul 10 21:29:05.552808 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.552779    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4/kube-rbac-proxy/0.log"
Jul 10 21:29:05.571331 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.571299    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4/kube-rbac-proxy-metric/0.log"
Jul 10 21:29:05.593424 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.593352    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4/prom-label-proxy/0.log"
Jul 10 21:29:05.613664 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.613632    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e80e7205-76fb-4c1a-9dcf-c387f7dfb1a4/init-config-reloader/0.log"
Jul 10 21:29:05.791976 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.791949    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-tz5ck_77245f92-3a87-45fc-8024-64ab9843d1e0/monitoring-plugin/0.log"
Jul 10 21:29:05.891810 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.891734    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dfncg_265baed6-9e3d-42d0-9250-a4129f1ee5d8/node-exporter/0.log"
Jul 10 21:29:05.911772 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.911745    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dfncg_265baed6-9e3d-42d0-9250-a4129f1ee5d8/kube-rbac-proxy/0.log"
Jul 10 21:29:05.933440 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:05.933415    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dfncg_265baed6-9e3d-42d0-9250-a4129f1ee5d8/init-textfile/0.log"
Jul 10 21:29:06.265251 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:06.265217    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-2gjnt_c184f503-61b0-4230-84e2-3ff6b2a19aa9/prometheus-operator/0.log"
Jul 10 21:29:06.282330 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:06.282282    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-2gjnt_c184f503-61b0-4230-84e2-3ff6b2a19aa9/kube-rbac-proxy/0.log"
Jul 10 21:29:06.399498 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:06.399467    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cbf9ddd68-4j7kn_2d43a49a-a3ef-4309-8bb0-f23362fa73aa/thanos-query/0.log"
Jul 10 21:29:06.419031 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:06.418985    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cbf9ddd68-4j7kn_2d43a49a-a3ef-4309-8bb0-f23362fa73aa/kube-rbac-proxy-web/0.log"
Jul 10 21:29:06.441052 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:06.440969    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cbf9ddd68-4j7kn_2d43a49a-a3ef-4309-8bb0-f23362fa73aa/kube-rbac-proxy/0.log"
Jul 10 21:29:06.460943 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:06.460910    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cbf9ddd68-4j7kn_2d43a49a-a3ef-4309-8bb0-f23362fa73aa/prom-label-proxy/0.log"
Jul 10 21:29:06.480390 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:06.480363    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cbf9ddd68-4j7kn_2d43a49a-a3ef-4309-8bb0-f23362fa73aa/kube-rbac-proxy-rules/0.log"
Jul 10 21:29:06.503092 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:06.503065    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cbf9ddd68-4j7kn_2d43a49a-a3ef-4309-8bb0-f23362fa73aa/kube-rbac-proxy-metrics/0.log"
Jul 10 21:29:07.609050 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:07.609015    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-wcxlk_eedc1267-4e6d-4dcf-b1eb-1232f2214f2d/networking-console-plugin/0.log"
Jul 10 21:29:08.190874 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.190838    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"]
Jul 10 21:29:08.199473 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.199432    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.206190 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.206160    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"]
Jul 10 21:29:08.219462 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.219437    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/2.log"
Jul 10 21:29:08.224043 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.223908    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-zxhjm_cc40c181-ad1c-4ef7-91cf-b35d8343f590/console-operator/3.log"
Jul 10 21:29:08.320323 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.320274    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-podres\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.320524 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.320432    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkkqq\" (UniqueName: \"kubernetes.io/projected/c6625dec-17b3-47ee-b7b2-0d5aa4395197-kube-api-access-tkkqq\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.320524 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.320488    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-proc\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.320692 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.320523    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-lib-modules\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.320692 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.320549    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-sys\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.421651 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.421604    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkkqq\" (UniqueName: \"kubernetes.io/projected/c6625dec-17b3-47ee-b7b2-0d5aa4395197-kube-api-access-tkkqq\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.421838 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.421669    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-proc\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.421838 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.421702    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-lib-modules\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.421838 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.421727    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-sys\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.421838 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.421798    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-podres\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.421838 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.421824    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-proc\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.422048 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.421850    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-sys\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.422048 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.421898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-lib-modules\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.422048 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.421930    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c6625dec-17b3-47ee-b7b2-0d5aa4395197-podres\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.430253 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.430220    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkkqq\" (UniqueName: \"kubernetes.io/projected/c6625dec-17b3-47ee-b7b2-0d5aa4395197-kube-api-access-tkkqq\") pod \"perf-node-gather-daemonset-k8t7t\" (UID: \"c6625dec-17b3-47ee-b7b2-0d5aa4395197\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.515316 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.515215    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:08.729127 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.728812    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"]
Jul 10 21:29:08.822503 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.822475    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c6955dd9d-cdmlh_b41a85dc-ea87-4401-a07d-c68cdb5231de/console/0.log"
Jul 10 21:29:08.858902 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:08.858860    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-p9l79_bb033ea7-58e3-46e6-93b5-ed1c1b11c632/download-server/0.log"
Jul 10 21:29:09.396056 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:09.396007    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-rgbms_b41db78f-c0d1-4449-9156-ce1651818206/volume-data-source-validator/0.log"
Jul 10 21:29:09.413585 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:09.413512    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t" event={"ID":"c6625dec-17b3-47ee-b7b2-0d5aa4395197","Type":"ContainerStarted","Data":"3b6fa2d245de10e3bea337b8b2393f2fdbfda03a37ccb98de58c2eaca6c4c661"}
Jul 10 21:29:09.413771 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:09.413613    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t" event={"ID":"c6625dec-17b3-47ee-b7b2-0d5aa4395197","Type":"ContainerStarted","Data":"bdd7b637d9f1bd72f7c311092962e8ac6a3e3a1ab2ea12c544b33a513fe16a31"}
Jul 10 21:29:09.413771 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:09.413647    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:09.432148 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:09.432098    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t" podStartSLOduration=1.4320816619999999 podStartE2EDuration="1.432081662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 21:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:29:09.430060124 +0000 UTC m=+3085.922856304" watchObservedRunningTime="2026-07-10 21:29:09.432081662 +0000 UTC m=+3085.924877843"
Jul 10 21:29:10.488890 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:10.488863    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ts8f9_2265ffde-452f-4d13-9350-f0ba6bc280a2/dns/0.log"
Jul 10 21:29:10.507113 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:10.507079    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ts8f9_2265ffde-452f-4d13-9350-f0ba6bc280a2/kube-rbac-proxy/0.log"
Jul 10 21:29:10.527753 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:10.527713    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9prbg_a4629e74-b03f-44c1-b4b6-c46c6954b27a/dns-node-resolver/0.log"
Jul 10 21:29:11.158271 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:11.158236    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zhv8j_77bf1e8e-d631-4017-8dd2-7e8a4a094022/node-ca/0.log"
Jul 10 21:29:12.293297 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:12.293264    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86d9dbc65b-5wdn8_c98ed95a-26fc-4dc8-880b-5c379f0d19b6/kube-auth-proxy/0.log"
Jul 10 21:29:13.187552 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:13.187518    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t8kvw_ff8e2022-8490-410b-968a-68c43e8d25f4/serve-healthcheck-canary/0.log"
Jul 10 21:29:13.647949 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:13.647919    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-xhsh5_73cf4177-2d24-4f8f-a408-53dac7359398/insights-operator/0.log"
Jul 10 21:29:13.648359 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:13.648310    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-xhsh5_73cf4177-2d24-4f8f-a408-53dac7359398/insights-operator/1.log"
Jul 10 21:29:13.724596 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:13.724551    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgsdg_50d32382-94d8-407b-aeef-4b0490d751bf/kube-rbac-proxy/0.log"
Jul 10 21:29:13.744691 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:13.744669    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgsdg_50d32382-94d8-407b-aeef-4b0490d751bf/exporter/0.log"
Jul 10 21:29:13.762932 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:13.762902    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgsdg_50d32382-94d8-407b-aeef-4b0490d751bf/extractor/0.log"
Jul 10 21:29:15.436814 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:15.436782    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-k8t7t"
Jul 10 21:29:16.540388 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:16.540350    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6558bffdbc-b2zgh_cc7d9625-4d68-4299-8ad0-82d8779118d7/manager/0.log"
Jul 10 21:29:16.561832 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:16.561794    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6558bffdbc-b2zgh_cc7d9625-4d68-4299-8ad0-82d8779118d7/copy-manifests/0.log"
Jul 10 21:29:18.151833 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:18.151750    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66b48cb7bf-qgn2z_2392acb8-240a-42ab-8c97-c282c0c64489/manager/0.log"
Jul 10 21:29:18.665110 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:18.665081    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86d9dbc65b-5wdn8_c98ed95a-26fc-4dc8-880b-5c379f0d19b6/kube-auth-proxy/0.log"
Jul 10 21:29:20.238994 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:20.238966    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86d9dbc65b-5wdn8_c98ed95a-26fc-4dc8-880b-5c379f0d19b6/kube-auth-proxy/0.log"
Jul 10 21:29:23.010781 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:23.010746    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-zn7ml_c8e66bf2-ca81-4e43-a8ee-6de156f4aee5/migrator/0.log"
Jul 10 21:29:23.028330 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:23.028304    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-zn7ml_c8e66bf2-ca81-4e43-a8ee-6de156f4aee5/graceful-termination/0.log"
Jul 10 21:29:24.410164 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.410127    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2bfwm_e8085ca0-9a5f-4ca5-b7f6-adbf9fe71900/kube-multus/0.log"
Jul 10 21:29:24.430715 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.430680    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4slff_5918c8f3-28f5-411f-ab01-e7fa1c9e117c/kube-multus-additional-cni-plugins/0.log"
Jul 10 21:29:24.447960 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.447935    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4slff_5918c8f3-28f5-411f-ab01-e7fa1c9e117c/egress-router-binary-copy/0.log"
Jul 10 21:29:24.465796 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.465771    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4slff_5918c8f3-28f5-411f-ab01-e7fa1c9e117c/cni-plugins/0.log"
Jul 10 21:29:24.482484 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.482456    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4slff_5918c8f3-28f5-411f-ab01-e7fa1c9e117c/bond-cni-plugin/0.log"
Jul 10 21:29:24.498622 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.498595    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4slff_5918c8f3-28f5-411f-ab01-e7fa1c9e117c/routeoverride-cni/0.log"
Jul 10 21:29:24.516864 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.516822    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4slff_5918c8f3-28f5-411f-ab01-e7fa1c9e117c/whereabouts-cni-bincopy/0.log"
Jul 10 21:29:24.536964 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.536934    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4slff_5918c8f3-28f5-411f-ab01-e7fa1c9e117c/whereabouts-cni/0.log"
Jul 10 21:29:24.925915 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.925886    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nbqvl_5436825a-b217-4c2d-a358-97173d8da40a/network-metrics-daemon/0.log"
Jul 10 21:29:24.940738 ip-10-0-141-228 kubenswrapper[2565]: I0710 21:29:24.940703    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nbqvl_5436825a-b217-4c2d-a358-97173d8da40a/kube-rbac-proxy/0.log"