May 20 15:29:12.326176 ip-10-0-138-16 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 20 15:29:12.326193 ip-10-0-138-16 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 20 15:29:12.326202 ip-10-0-138-16 systemd[1]: kubelet.service: Failed with result 'resources'.
May 20 15:29:12.326502 ip-10-0-138-16 systemd[1]: Failed to start Kubernetes Kubelet.
May 20 15:29:21.693141 ip-10-0-138-16 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
May 20 15:29:21.693155 ip-10-0-138-16 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 08b1b454a86343cc95d881b60cce7d7c --
May 20 15:31:30.183096 ip-10-0-138-16 systemd[1]: Starting Kubernetes Kubelet...
May 20 15:31:30.690485 ip-10-0-138-16 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 20 15:31:30.690485 ip-10-0-138-16 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 20 15:31:30.690485 ip-10-0-138-16 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 20 15:31:30.690485 ip-10-0-138-16 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 20 15:31:30.691146 ip-10-0-138-16 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 20 15:31:30.692933 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.692849    2566 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 20 15:31:30.699144 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699129    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 20 15:31:30.699144 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699143    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699147    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699151    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699154    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699156    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699159    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699164    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699167    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699171    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699173    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699177    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699179    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699182    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699185    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699187    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699189    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699192    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699194    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699197    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699199    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 20 15:31:30.699207 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699202    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699204    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699207    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699210    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699213    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699216    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699219    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699221    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699224    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699226    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699228    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699231    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699234    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699237    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699239    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699242    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699244    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699247    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699249    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699252    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 20 15:31:30.699682 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699254    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699257    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699259    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699262    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699264    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699266    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699269    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699271    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699273    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699276    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699278    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699280    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699283    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699285    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699288    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699291    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699294    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699296    2566 feature_gate.go:328] unrecognized feature gate: Example
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699299    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699301    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 20 15:31:30.700380 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699303    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699306    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699308    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699311    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699313    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699316    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699318    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699320    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699324    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699328    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699332    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699336    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699339    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699342    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699362    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699366    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699369    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699373    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699376    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 20 15:31:30.701005 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699379    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699382    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699384    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699387    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699389    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.699392    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701189    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701202    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701208    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701211    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701214    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701219    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701224    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701227    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701230    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701233    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701236    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701238    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701242    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 20 15:31:30.701472 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701245    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701247    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701250    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701252    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701255    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701257    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701260    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701262    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701265    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701267    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701270    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701272    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701276    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701280    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701282    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701291    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701294    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701297    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701299    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701302    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 20 15:31:30.701946 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701306    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701309    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701312    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701314    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701317    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701320    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701323    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701326    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701329    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701331    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701334    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701336    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701339    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701341    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701344    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701347    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701349    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701352    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701354    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701357    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 20 15:31:30.702433 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701359    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701362    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701364    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701367    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701370    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701372    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701375    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701377    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701380    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701383    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701386    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701388    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701391    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701393    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701396    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701398    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701401    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701404    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701406    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 20 15:31:30.703023 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701409    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701411    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701414    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701416    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701418    2566 feature_gate.go:328] unrecognized feature gate: Example
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701421    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701430    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701433    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701435    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701438    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701440    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701443    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701446    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.701450    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701522    2566 flags.go:64] FLAG: --address="0.0.0.0"
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701529    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701539    2566 flags.go:64] FLAG: --anonymous-auth="true"
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701543    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701550    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701553    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701557    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 20 15:31:30.703500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701562    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701565    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701569    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701572    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701577    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701580    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701583    2566 flags.go:64] FLAG: --cgroup-root=""
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701586    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701589    2566 flags.go:64] FLAG: --client-ca-file=""
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701592    2566 flags.go:64] FLAG: --cloud-config=""
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701595    2566 flags.go:64] FLAG: --cloud-provider="external"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.701597    2566 flags.go:64] FLAG: --cluster-dns="[]"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702219    2566 flags.go:64] FLAG: --cluster-domain=""
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702223    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702226    2566 flags.go:64] FLAG: --config-dir=""
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702229    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702233    2566 flags.go:64] FLAG: --container-log-max-files="5"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702237    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702246    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702249    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702252    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702255    2566 flags.go:64] FLAG: --contention-profiling="false"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702257    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702260    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702263    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
May 20 15:31:30.704021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702266    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702270    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702273    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702276    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702279    2566 flags.go:64] FLAG: --enable-load-reader="false"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702283    2566 flags.go:64] FLAG: --enable-server="true"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702285    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702290    2566 flags.go:64] FLAG: --event-burst="100"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702294    2566 flags.go:64] FLAG: --event-qps="50"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702297    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702300    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702303    2566 flags.go:64] FLAG: --eviction-hard=""
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702307    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702310    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702313    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702316    2566 flags.go:64] FLAG: --eviction-soft=""
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702319    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702322    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702324    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702327    2566 flags.go:64] FLAG: --experimental-mounter-path=""
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702330    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702333    2566 flags.go:64] FLAG: --fail-swap-on="true"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702335    2566 flags.go:64] FLAG: --feature-gates=""
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702339    2566 flags.go:64] FLAG: --file-check-frequency="20s"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702342    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 20 15:31:30.704636 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702345    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702354    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702357    2566 flags.go:64] FLAG: --healthz-port="10248"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702360    2566 flags.go:64] FLAG: --help="false"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702363    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-138-16.ec2.internal"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702366    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702368    2566 flags.go:64] FLAG: --http-check-frequency="20s"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702371    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702375    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702378    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702381    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702383    2566 flags.go:64] FLAG: --image-service-endpoint=""
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702386    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702389    2566 flags.go:64] FLAG: --kube-api-burst="100"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702391    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702395    2566 flags.go:64] FLAG: --kube-api-qps="50"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702398    2566 flags.go:64] FLAG: --kube-reserved=""
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702400    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702403    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702407    2566 flags.go:64] FLAG: --kubelet-cgroups=""
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702410    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702412    2566 flags.go:64] FLAG: --lock-file=""
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702415    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702418    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
May 20 15:31:30.705301 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702421    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702426    2566 flags.go:64] FLAG: --log-json-split-stream="false"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702429    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702432    2566 flags.go:64] FLAG: --log-text-split-stream="false"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702435    2566 flags.go:64] FLAG: --logging-format="text"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702438    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702441    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702444    2566 flags.go:64] FLAG: --manifest-url=""
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702447    2566 flags.go:64] FLAG: --manifest-url-header=""
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702451    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702459    2566 flags.go:64] FLAG: --max-open-files="1000000"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702464    2566 flags.go:64] FLAG: --max-pods="110"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702466    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702469    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702472    2566 flags.go:64] FLAG: --memory-manager-policy="None"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702475    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702478    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702480    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702483    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702491    2566 flags.go:64] FLAG: --node-status-max-images="50"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702495    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702498    2566 flags.go:64] FLAG: --oom-score-adj="-999"
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702501    2566 flags.go:64] FLAG: --pod-cidr=""
May 20 15:31:30.705901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702504    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702509    2566 flags.go:64] FLAG: --pod-manifest-path=""
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702512    2566 flags.go:64] FLAG: --pod-max-pids="-1"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702515    2566 flags.go:64] FLAG: --pods-per-core="0"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702518    2566 flags.go:64] FLAG: --port="10250"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702522    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702524    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00763c5902543aca5"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702528    2566 flags.go:64] FLAG: --qos-reserved=""
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702530    2566 flags.go:64] FLAG: --read-only-port="10255"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702533    2566 flags.go:64] FLAG: --register-node="true"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702536    2566 flags.go:64] FLAG: --register-schedulable="true"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702539    2566 flags.go:64] FLAG: --register-with-taints=""
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702542    2566 flags.go:64] FLAG: --registry-burst="10"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702545    2566 flags.go:64] FLAG: --registry-qps="5"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702548    2566 flags.go:64] FLAG: --reserved-cpus=""
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702551    2566 flags.go:64] FLAG: --reserved-memory=""
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702554    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702557    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702560    2566 flags.go:64] FLAG: --rotate-certificates="false"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702563    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702571    2566 flags.go:64] FLAG: --runonce="false"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702574    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702577    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702580    2566 flags.go:64] FLAG: --seccomp-default="false"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702583    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702586    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 20 15:31:30.706498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702589    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702592    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702595    2566 flags.go:64] FLAG: --storage-driver-password="root"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702597    2566 flags.go:64] FLAG: --storage-driver-secure="false"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702600    2566 flags.go:64] FLAG: --storage-driver-table="stats"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702603    2566 flags.go:64] FLAG: --storage-driver-user="root"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702606    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702609    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702611    2566 flags.go:64] FLAG: --system-cgroups=""
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702614    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702620    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702623    2566 flags.go:64] FLAG: --tls-cert-file=""
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702626    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702631    2566 flags.go:64] FLAG: --tls-min-version=""
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702634    2566 flags.go:64] FLAG: --tls-private-key-file=""
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702637    2566 flags.go:64] FLAG: --topology-manager-policy="none"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702639    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702642    2566 flags.go:64] FLAG: --topology-manager-scope="container"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702645    2566 flags.go:64] FLAG: --v="2"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702650    2566 flags.go:64] FLAG: --version="false"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702653    2566 flags.go:64] FLAG: --vmodule=""
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702658    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.702661    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702767    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 20 15:31:30.707145 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702770    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702774    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702776    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702780    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702782    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702785    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702790    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702793    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702795    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702798    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702800    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702803    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702806    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702808    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702811    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702813    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702816    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702818    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702823    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 20 15:31:30.707722 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702827    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702830    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702832    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702835    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702838    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702840    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702843    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702845    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702847    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702850    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702852    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702855    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702857    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702860    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702862    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702865    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702867    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702870    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702873    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702876    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 20 15:31:30.708227 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702879    2566 feature_gate.go:328] unrecognized feature gate: Example
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702881    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702883    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702886    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702888    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702891    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702893    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702896    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702898    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702900    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702903    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702907    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702910    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702912    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702915    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702917    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702919    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702922    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702924    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702927    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702929    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 20 15:31:30.708890 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702932    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702934    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702937    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702939    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702942    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702944    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702947    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702949    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702951    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702954    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702958    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702960    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702963    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702965    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702968    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702970    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702973    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702976    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702979    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 20 15:31:30.709789 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702981    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702985    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702989    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702993    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702996    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.702999    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.703004    2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.710552    2566 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.710575    2566 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710646    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710654    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710659    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 20 15:31:30.710655 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710664    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710669    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710674    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710679    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710683    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710687    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710692    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710696    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710700    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710704    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710709    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710714    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710718    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710722    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710729    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710735    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710740    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710768    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710772    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 20 15:31:30.711222 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710777    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710781    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710785    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710789    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710793    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710797    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710801    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710805    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710809    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710813    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710817    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710821    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710826    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710830    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710834    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710839    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710843    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710847    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710852    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 20 15:31:30.711954 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710856    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710860    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710864    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710868    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710872    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710877    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710881    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710886    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710889    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710894    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710898    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710904    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710910    2566 feature_gate.go:328] unrecognized feature gate: Example
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710914    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710919    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710923    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710927    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710931    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710935    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710939    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 20 15:31:30.712435 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710943    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710947    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710951    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710955    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710959    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710964    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710968    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710972    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710977    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710981    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710985    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710989    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710993    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.710997    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711001    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711005    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711009    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711013    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711017    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711021    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 20 15:31:30.713017 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711025    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711029    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711034    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711037    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711041    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.711050    2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711210    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711217    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711222    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711226    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711230    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711235    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711239    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711243    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711247    2566 feature_gate.go:328] unrecognized feature gate: Example
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711251    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 20 15:31:30.713685 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711255    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711259    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711264    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711268    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711272    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711277    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711281    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711285    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711289    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711293    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711297    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711301    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711306    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711313    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711318    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711324    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711328    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711333    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711337    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 20 15:31:30.714382 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711342    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711346    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711351    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711355    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711359    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711364    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711368    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711373    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711377    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711381    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711386    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711390    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711394    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711398    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711403    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711408    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711412    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711416    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711420    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711424    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 20 15:31:30.715116 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711428    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711432    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711436    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711440    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711445    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711448    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711452    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711456    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711461    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711466    2566 feature_gate.go:328] unrecognized feature gate: Example2
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711470    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711474    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711479    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711483    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711487    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711491    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711495    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711499    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711503    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711507    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 20 15:31:30.715729 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711511    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711514    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711518    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711523    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711527    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711531    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711535    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711539    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711543    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711547    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711551    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711555    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711559    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711562    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711566    2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711570    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 20 15:31:30.716331 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:30.711577    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 20 15:31:30.716718 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.711585    2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 20 15:31:30.716718 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.712560    2566 server.go:962] "Client rotation is on, will bootstrap in background"
May 20 15:31:30.716718 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.715630    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 20 15:31:30.716861 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.716808    2566 server.go:1019] "Starting client certificate rotation"
May 20 15:31:30.716928 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.716910    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 20 15:31:30.716962 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.716948    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 20 15:31:30.744610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.744584    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 20 15:31:30.746771 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.746735    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 20 15:31:30.766383 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.766363    2566 log.go:25] "Validated CRI v1 runtime API"
May 20 15:31:30.773171 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.773154    2566 log.go:25] "Validated CRI v1 image API"
May 20 15:31:30.775567 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.775550    2566 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 20 15:31:30.778187 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.778169    2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 20 15:31:30.780665 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.780645    2566 fs.go:135] Filesystem UUIDs: map[276a2215-06c4-414c-b524-429ecc40431c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ef80ed89-c639-4204-9326-297e9b2b9515:/dev/nvme0n1p4]
May 20 15:31:30.780709 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.780666    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 20 15:31:30.784968 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.784858    2566 manager.go:217] Machine: {Timestamp:2026-05-20 15:31:30.784363835 +0000 UTC m=+0.468233165 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104926 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25c8bead45cf88c2d197909d532a18 SystemUUID:ec25c8be-ad45-cf88-c2d1-97909d532a18 BootID:08b1b454-a863-43cc-95d8-81b60cce7d7c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0d:9f:68:d8:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0d:9f:68:d8:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:cc:79:b1:b2:eb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 20 15:31:30.785869 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.785858    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 20 15:31:30.785988 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.785975    2566 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 20 15:31:30.788348 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.788321    2566 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 20 15:31:30.788486 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.788352    2566 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-16.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 20 15:31:30.788531 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.788496    2566 topology_manager.go:138] "Creating topology manager with none policy"
May 20 15:31:30.788531 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.788505    2566 container_manager_linux.go:306] "Creating device plugin manager"
May 20 15:31:30.788531 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.788512    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 20 15:31:30.789443 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.789432    2566 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 20 15:31:30.791259 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.791249    2566 state_mem.go:36] "Initialized new in-memory state store"
May 20 15:31:30.791380 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.791371    2566 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 20 15:31:30.794741 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.794729    2566 kubelet.go:491] "Attempting to sync node with API server"
May 20 15:31:30.794796 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.794757    2566 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 20 15:31:30.794796 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.794770    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 20 15:31:30.794796 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.794780    2566 kubelet.go:397] "Adding apiserver pod source"
May 20 15:31:30.794796 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.794793    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 20 15:31:30.796068 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.796056    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 20 15:31:30.796112 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.796076    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 20 15:31:30.799539 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.799521    2566 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 20 15:31:30.801923 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.801910    2566 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 20 15:31:30.802903 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802893    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 20 15:31:30.802939 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802910    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 20 15:31:30.802939 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802916    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 20 15:31:30.802939 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802932    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 20 15:31:30.803017 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802941    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 20 15:31:30.803017 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802947    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 20 15:31:30.803017 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802953    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 20 15:31:30.803017 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802959    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 20 15:31:30.803017 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802966    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 20 15:31:30.803017 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802972    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 20 15:31:30.803017 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.802983    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 20 15:31:30.803456 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.803448    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 20 15:31:30.805623 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.805612    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 20 15:31:30.805666 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.805625    2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 20 15:31:30.808727 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.808713    2566 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 20 15:31:30.808804 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.808766    2566 server.go:1295] "Started kubelet"
May 20 15:31:30.808891 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.808846    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 20 15:31:30.808951 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.808899    2566 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 20 15:31:30.809002 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.808978    2566 server_v1.go:47] "podresources" method="list" useActivePods=true
May 20 15:31:30.812986 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.812961    2566 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 20 15:31:30.813363 ip-10-0-138-16 systemd[1]: Started Kubernetes Kubelet.
May 20 15:31:30.815711 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.814294    2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 20 15:31:30.815711 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.814393    2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 20 15:31:30.815711 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.814398    2566 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-16.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 20 15:31:30.815711 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.814551    2566 server.go:317] "Adding debug handlers to kubelet server"
May 20 15:31:30.819873 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.818709    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-16.ec2.internal.18b14fd1ebabe5d7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-16.ec2.internal,UID:ip-10-0-138-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-16.ec2.internal,},FirstTimestamp:2026-05-20 15:31:30.808726999 +0000 UTC m=+0.492596330,LastTimestamp:2026-05-20 15:31:30.808726999 +0000 UTC m=+0.492596330,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-16.ec2.internal,}"
May 20 15:31:30.823274 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.823251    2566 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 20 15:31:30.824731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.824717    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 20 15:31:30.824827 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.824737    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 20 15:31:30.824827 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.824788    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zg5ct"
May 20 15:31:30.824930 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.824857    2566 volume_manager.go:295] "The desired_state_of_world populator starts"
May 20 15:31:30.824930 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.824869    2566 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 20 15:31:30.825020 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.824924    2566 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 20 15:31:30.825020 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.824980    2566 reconstruct.go:97] "Volume reconstruction finished"
May 20 15:31:30.825020 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.824991    2566 reconciler.go:26] "Reconciler: start to sync state"
May 20 15:31:30.825508 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.825489    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:30.825508 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.825492    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 20 15:31:30.825634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.825517    2566 factory.go:55] Registering systemd factory
May 20 15:31:30.825634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.825528    2566 factory.go:223] Registration of the systemd container factory successfully
May 20 15:31:30.825790 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.825777    2566 factory.go:153] Registering CRI-O factory
May 20 15:31:30.825864 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.825794    2566 factory.go:223] Registration of the crio container factory successfully
May 20 15:31:30.825864 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.825808    2566 factory.go:103] Registering Raw factory
May 20 15:31:30.825864 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.825816    2566 manager.go:1196] Started watching for new ooms in manager
May 20 15:31:30.826311 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.826299    2566 manager.go:319] Starting recovery of all containers
May 20 15:31:30.827019 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.826814    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 20 15:31:30.827019 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.826989    2566 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 20 15:31:30.833355 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.833336    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zg5ct"
May 20 15:31:30.836018 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.835993    2566 manager.go:324] Recovery completed
May 20 15:31:30.840334 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.840309    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 20 15:31:30.842411 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.842397    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientMemory"
May 20 15:31:30.842470 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.842425    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasNoDiskPressure"
May 20 15:31:30.842470 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.842441    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientPID"
May 20 15:31:30.842920 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.842907    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 20 15:31:30.842920 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.842919    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 20 15:31:30.843005 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.842933    2566 state_mem.go:36] "Initialized new in-memory state store"
May 20 15:31:30.845669 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.845654    2566 policy_none.go:49] "None policy: Start"
May 20 15:31:30.845761 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.845673    2566 memory_manager.go:186] "Starting memorymanager" policy="None"
May 20 15:31:30.845761 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.845683    2566 state_mem.go:35] "Initializing new in-memory state store"
May 20 15:31:30.892310 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.892291    2566 manager.go:341] "Starting Device Plugin manager"
May 20 15:31:30.899605 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.892328    2566 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 20 15:31:30.899605 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.892341    2566 server.go:85] "Starting device plugin registration server"
May 20 15:31:30.899605 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.892568    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
May 20 15:31:30.899605 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.892581    2566 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 20 15:31:30.899605 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.892675    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 20 15:31:30.899605 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.892771    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 20 15:31:30.899605 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.892781    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 20 15:31:30.899605 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.893216    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 20 15:31:30.899605 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.893251    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:30.926100 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.926076    2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 20 15:31:30.927419 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.927398    2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 20 15:31:30.927514 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.927423    2566 status_manager.go:230] "Starting to sync pod status with apiserver"
May 20 15:31:30.927514 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.927445    2566 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 20 15:31:30.927514 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.927456    2566 kubelet.go:2451] "Starting kubelet main sync loop"
May 20 15:31:30.927647 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:30.927539    2566 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 20 15:31:30.930767 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.930734    2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 20 15:31:30.993196 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.993133    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 20 15:31:30.993986 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.993971    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientMemory"
May 20 15:31:30.994040 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.994004    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasNoDiskPressure"
May 20 15:31:30.994040 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.994020    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientPID"
May 20 15:31:30.994105 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:30.994042    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-16.ec2.internal"
May 20 15:31:31.004490 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.004473    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-16.ec2.internal"
May 20 15:31:31.004552 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.004493    2566 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-16.ec2.internal\": node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.028041 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.028020    2566 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal"]
May 20 15:31:31.028116 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.028093    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 20 15:31:31.029068 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.029038    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientMemory"
May 20 15:31:31.029068 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.029070    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasNoDiskPressure"
May 20 15:31:31.029231 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.029084    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientPID"
May 20 15:31:31.030221 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.030206    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 20 15:31:31.030433 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.030399    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.030484 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.030462    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 20 15:31:31.030864 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.030848    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientMemory"
May 20 15:31:31.030955 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.030878    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasNoDiskPressure"
May 20 15:31:31.030955 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.030895    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientPID"
May 20 15:31:31.031160 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.031137    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientMemory"
May 20 15:31:31.031221 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.031167    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasNoDiskPressure"
May 20 15:31:31.031221 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.031183    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientPID"
May 20 15:31:31.031915 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.031899    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.031976 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.031923    2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 20 15:31:31.032548 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.032534    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientMemory"
May 20 15:31:31.032612 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.032558    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasNoDiskPressure"
May 20 15:31:31.032612 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.032568    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeHasSufficientPID"
May 20 15:31:31.038121 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.038105    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.050299 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.050271    2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-16.ec2.internal\" not found" node="ip-10-0-138-16.ec2.internal"
May 20 15:31:31.054154 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.054130    2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-16.ec2.internal\" not found" node="ip-10-0-138-16.ec2.internal"
May 20 15:31:31.126380 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.126351    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4fa92cd324f2b8adbc6bdb5146df37d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal\" (UID: \"b4fa92cd324f2b8adbc6bdb5146df37d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.126380 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.126380    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4fa92cd324f2b8adbc6bdb5146df37d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal\" (UID: \"b4fa92cd324f2b8adbc6bdb5146df37d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.126565 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.126397    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/49adea86e64f32c62411eb9c86b01363-config\") pod \"kube-apiserver-proxy-ip-10-0-138-16.ec2.internal\" (UID: \"49adea86e64f32c62411eb9c86b01363\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.138822 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.138803    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.226971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.226944    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4fa92cd324f2b8adbc6bdb5146df37d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal\" (UID: \"b4fa92cd324f2b8adbc6bdb5146df37d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.227070 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.226977    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4fa92cd324f2b8adbc6bdb5146df37d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal\" (UID: \"b4fa92cd324f2b8adbc6bdb5146df37d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.227070 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.226996    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/49adea86e64f32c62411eb9c86b01363-config\") pod \"kube-apiserver-proxy-ip-10-0-138-16.ec2.internal\" (UID: \"49adea86e64f32c62411eb9c86b01363\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.227070 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.227039    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4fa92cd324f2b8adbc6bdb5146df37d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal\" (UID: \"b4fa92cd324f2b8adbc6bdb5146df37d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.227070 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.227050    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4fa92cd324f2b8adbc6bdb5146df37d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal\" (UID: \"b4fa92cd324f2b8adbc6bdb5146df37d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.227070 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.227044    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/49adea86e64f32c62411eb9c86b01363-config\") pod \"kube-apiserver-proxy-ip-10-0-138-16.ec2.internal\" (UID: \"49adea86e64f32c62411eb9c86b01363\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.239067 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.239032    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.339929 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.339868    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.353045 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.353022    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.356822 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.356806    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal"
May 20 15:31:31.440720 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.440686    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.541204 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.541177    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.641778 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.641686    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.718916 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.718886    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 20 15:31:31.719480 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.719022    2566 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 20 15:31:31.742482 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.742321    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.817911 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:31.817870    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4fa92cd324f2b8adbc6bdb5146df37d.slice/crio-a697b1ac9e9014df6f9f527c13c71eec8318de5af3745bc352c0373442ee083e WatchSource:0}: Error finding container a697b1ac9e9014df6f9f527c13c71eec8318de5af3745bc352c0373442ee083e: Status 404 returned error can't find the container with id a697b1ac9e9014df6f9f527c13c71eec8318de5af3745bc352c0373442ee083e
May 20 15:31:31.818102 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:31.818088    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49adea86e64f32c62411eb9c86b01363.slice/crio-751c9ad6b0f7cdda995d1589c22a3ef709b19f7a7a21b97b2cd965b4978a740e WatchSource:0}: Error finding container 751c9ad6b0f7cdda995d1589c22a3ef709b19f7a7a21b97b2cd965b4978a740e: Status 404 returned error can't find the container with id 751c9ad6b0f7cdda995d1589c22a3ef709b19f7a7a21b97b2cd965b4978a740e
May 20 15:31:31.819538 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.819423    2566 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 20 15:31:31.823601 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.823586    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 20 15:31:31.824897 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.824883    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 20 15:31:31.836277 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.836251    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-19 15:26:30 +0000 UTC" deadline="2027-12-11 13:52:29.793982973 +0000 UTC"
May 20 15:31:31.836277 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.836275    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13678h20m57.957710926s"
May 20 15:31:31.838048 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.838032    2566 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 20 15:31:31.843324 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.843304    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:31.850401 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.850386    2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 20 15:31:31.864665 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.864646    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6774p"
May 20 15:31:31.874292 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.874271    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6774p"
May 20 15:31:31.930945 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.930836    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal" event={"ID":"b4fa92cd324f2b8adbc6bdb5146df37d","Type":"ContainerStarted","Data":"a697b1ac9e9014df6f9f527c13c71eec8318de5af3745bc352c0373442ee083e"}
May 20 15:31:31.931729 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:31.931709    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal" event={"ID":"49adea86e64f32c62411eb9c86b01363","Type":"ContainerStarted","Data":"751c9ad6b0f7cdda995d1589c22a3ef709b19f7a7a21b97b2cd965b4978a740e"}
May 20 15:31:31.943884 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:31.943857    2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-16.ec2.internal\" not found"
May 20 15:31:32.009641 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.009621    2566 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 20 15:31:32.025055 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.025028    2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal"
May 20 15:31:32.042037 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.042015    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 20 15:31:32.043115 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.043100    2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal"
May 20 15:31:32.052868 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.052849    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 20 15:31:32.651718 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.651687    2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 20 15:31:32.795794 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.795763    2566 apiserver.go:52] "Watching apiserver"
May 20 15:31:32.805598 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.805574    2566 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 20 15:31:32.806819 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.806781    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-gj2r6","kube-system/konnectivity-agent-b5l6p","kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65","openshift-cluster-node-tuning-operator/tuned-hvk6w","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal","openshift-multus/multus-56bdg","openshift-network-diagnostics/network-check-target-mrqh2","openshift-network-operator/iptables-alerter-lc2zw","openshift-image-registry/node-ca-w8bpd","openshift-multus/multus-additional-cni-plugins-bhg2x","openshift-multus/network-metrics-daemon-2grdq"]
May 20 15:31:32.809239 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.809219    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.809556 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.809537    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:32.810686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.810647    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.812542    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.812934    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.812962    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-764gr\""
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813018    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813047    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813064    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813233    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813328    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813459    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 20 15:31:32.813622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813522    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 20 15:31:32.814193 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813719    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-55f6w\""
May 20 15:31:32.814193 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813946    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pt4bm\""
May 20 15:31:32.814193 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813990    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 20 15:31:32.814193 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.814118    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 20 15:31:32.814391 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.813946    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 20 15:31:32.814726 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.814707    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-56bdg"
May 20 15:31:32.815127 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.815106    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 20 15:31:32.815210 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.815139    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ttprt\""
May 20 15:31:32.815359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.815342    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 20 15:31:32.816777 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.816064    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:32.816777 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:32.816125    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:32.817148 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.816909    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 20 15:31:32.817148 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.817123    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xtdrr\""
May 20 15:31:32.817253 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.817198    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 20 15:31:32.817302 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.817269    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:32.817355 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.817333    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 20 15:31:32.817631 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.817619    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 20 15:31:32.818350 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.818332    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.821401 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.820275    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j66cd\""
May 20 15:31:32.821401 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.820536    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 20 15:31:32.821401 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.820546    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 20 15:31:32.821401 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.820809    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 20 15:31:32.821401 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.821314    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 20 15:31:32.821990 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.821971    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rjzrw\""
May 20 15:31:32.822102 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.822092    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 20 15:31:32.822307 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.822291    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 20 15:31:32.822360 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.822311    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.823779 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.823742    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:32.823870 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:32.823828    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:32.824681 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.824660    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 20 15:31:32.824775 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.824686    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 20 15:31:32.824775 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.824768    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zx8cc\""
May 20 15:31:32.825818 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.825807    2566 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 20 15:31:32.837052 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837031    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-cni-netd\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.837143 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837074    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-registration-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.837143 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837103    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-etc-openvswitch\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.837143 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837130    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgtrt\" (UniqueName: \"kubernetes.io/projected/0e962466-c4aa-4d88-b80a-1827a1583042-kube-api-access-xgtrt\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.837302 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837153    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.837302 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837205    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-sys-fs\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.837302 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837229    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-systemd\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.837302 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837253    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-sys\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.837302 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837278    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f11105f6-69d8-452b-9704-a4a7922fa418-agent-certs\") pod \"konnectivity-agent-b5l6p\" (UID: \"f11105f6-69d8-452b-9704-a4a7922fa418\") " pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:32.837534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837302    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-sysconfig\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.837534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837325    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-lib-modules\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.837534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837348    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccp5c\" (UniqueName: \"kubernetes.io/projected/e89102ba-1b0e-47d7-8bae-1f53a65984c6-kube-api-access-ccp5c\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.837534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837370    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:32.837534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837394    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-system-cni-dir\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.837534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837426    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-os-release\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.837534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837449    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:32.837534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-kubelet\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.837534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837521    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0e962466-c4aa-4d88-b80a-1827a1583042-env-overrides\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837548    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-system-cni-dir\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837576    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-os-release\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837601    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837655    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0b256b7b-d193-4847-a3c7-c0868823bbae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837711    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-systemd-units\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0e962466-c4aa-4d88-b80a-1827a1583042-ovnkube-config\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837804    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-run\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837834    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6-host-slash\") pod \"iptables-alerter-lc2zw\" (UID: \"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6\") " pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837891    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-run-multus-certs\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.837931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837924    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjv7m\" (UniqueName: \"kubernetes.io/projected/838a057a-e096-49a9-a086-9202ea786b05-kube-api-access-xjv7m\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837951    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-log-socket\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-node-log\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.837997    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxh7v\" (UniqueName: \"kubernetes.io/projected/2795d99a-b1c0-4151-b90a-0d6ce3e19488-kube-api-access-zxh7v\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838019    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-multus-cni-dir\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838042    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-run-netns\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838063    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-multus-conf-dir\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838085    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-cnibin\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838107    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf-host\") pod \"node-ca-w8bpd\" (UID: \"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf\") " pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838155    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-socket-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-var-lib-kubelet\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838230    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ae40396-0221-4852-a40a-ec38478dad13-cni-binary-copy\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.838287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838270    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-hostroot\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838307    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-sysctl-conf\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838356    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-host\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-etc-kubernetes\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-slash\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838443    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-run-ovn\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838468    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-etc-selinux\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838490    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-tuned\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838513    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-var-lib-kubelet\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838535    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b256b7b-d193-4847-a3c7-c0868823bbae-cni-binary-copy\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-cni-bin\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838598    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f11105f6-69d8-452b-9704-a4a7922fa418-konnectivity-ca\") pod \"konnectivity-agent-b5l6p\" (UID: \"f11105f6-69d8-452b-9704-a4a7922fa418\") " pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838621    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47ln4\" (UniqueName: \"kubernetes.io/projected/b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6-kube-api-access-47ln4\") pod \"iptables-alerter-lc2zw\" (UID: \"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6\") " pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838655    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ae40396-0221-4852-a40a-ec38478dad13-multus-daemon-config\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838693    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnsqx\" (UniqueName: \"kubernetes.io/projected/7ae40396-0221-4852-a40a-ec38478dad13-kube-api-access-mnsqx\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.838714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838716    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf-serviceca\") pod \"node-ca-w8bpd\" (UID: \"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf\") " pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838764    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0b256b7b-d193-4847-a3c7-c0868823bbae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838789    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-modprobe-d\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838811    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-cnibin\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838830    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-run-k8s-cni-cncf-io\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838846    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-var-lib-cni-bin\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838861    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-var-lib-openvswitch\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838876    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-run-ovn-kubernetes\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfzdn\" (UniqueName: \"kubernetes.io/projected/20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf-kube-api-access-bfzdn\") pod \"node-ca-w8bpd\" (UID: \"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf\") " pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838906    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgn7q\" (UniqueName: \"kubernetes.io/projected/0b256b7b-d193-4847-a3c7-c0868823bbae-kube-api-access-rgn7q\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838920    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-run-systemd\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838934    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-run-openvswitch\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838970    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.838988    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0e962466-c4aa-4d88-b80a-1827a1583042-ovnkube-script-lib\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.839001    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-kubernetes\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.839019    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e89102ba-1b0e-47d7-8bae-1f53a65984c6-tmp\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.839273 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.839041    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-var-lib-cni-multus\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.839895 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.839064    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-run-netns\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.839895 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.839087    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e962466-c4aa-4d88-b80a-1827a1583042-ovn-node-metrics-cert\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.839895 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.839111    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-device-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.839895 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.839135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-sysctl-d\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.839895 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.839163    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6-iptables-alerter-script\") pod \"iptables-alerter-lc2zw\" (UID: \"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6\") " pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:32.839895 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.839188    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-multus-socket-dir-parent\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.875827 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.875778    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-19 15:26:31 +0000 UTC" deadline="2027-11-29 10:24:27.312672809 +0000 UTC"
May 20 15:31:32.875947 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.875828    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13386h52m54.436849085s"
May 20 15:31:32.940143 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940074    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf-host\") pod \"node-ca-w8bpd\" (UID: \"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf\") " pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.940143 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940106    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-socket-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.940143 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940128    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-var-lib-kubelet\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940152    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ae40396-0221-4852-a40a-ec38478dad13-cni-binary-copy\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940177    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf-host\") pod \"node-ca-w8bpd\" (UID: \"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf\") " pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940216    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-hostroot\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940226    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-var-lib-kubelet\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940242    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-sysctl-conf\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940273    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-socket-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940281    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-host\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940321    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-host\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940331    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-etc-kubernetes\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940356    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-slash\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940363    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-hostroot\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940373    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-run-ovn\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940373    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-sysctl-conf\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940401    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-etc-kubernetes\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940406    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-etc-selinux\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.940404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940405    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-run-ovn\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940433    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-tuned\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940461    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-var-lib-kubelet\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940485    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b256b7b-d193-4847-a3c7-c0868823bbae-cni-binary-copy\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940436    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-slash\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940509    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-cni-bin\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940532    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-var-lib-kubelet\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940492    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-etc-selinux\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940545    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f11105f6-69d8-452b-9704-a4a7922fa418-konnectivity-ca\") pod \"konnectivity-agent-b5l6p\" (UID: \"f11105f6-69d8-452b-9704-a4a7922fa418\") " pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940578    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47ln4\" (UniqueName: \"kubernetes.io/projected/b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6-kube-api-access-47ln4\") pod \"iptables-alerter-lc2zw\" (UID: \"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6\") " pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940602    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ae40396-0221-4852-a40a-ec38478dad13-multus-daemon-config\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940626    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mnsqx\" (UniqueName: \"kubernetes.io/projected/7ae40396-0221-4852-a40a-ec38478dad13-kube-api-access-mnsqx\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940649    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf-serviceca\") pod \"node-ca-w8bpd\" (UID: \"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf\") " pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940675    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0b256b7b-d193-4847-a3c7-c0868823bbae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940697    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ae40396-0221-4852-a40a-ec38478dad13-cni-binary-copy\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940703    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-modprobe-d\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940728    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-cnibin\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940714    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 20 15:31:32.941161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940773    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-run-k8s-cni-cncf-io\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940798    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-var-lib-cni-bin\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940822    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-var-lib-openvswitch\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940843    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-run-ovn-kubernetes\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940863    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bfzdn\" (UniqueName: \"kubernetes.io/projected/20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf-kube-api-access-bfzdn\") pod \"node-ca-w8bpd\" (UID: \"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf\") " pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940886    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rgn7q\" (UniqueName: \"kubernetes.io/projected/0b256b7b-d193-4847-a3c7-c0868823bbae-kube-api-access-rgn7q\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940907    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-run-systemd\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940929    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-run-openvswitch\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940955    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940980    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0e962466-c4aa-4d88-b80a-1827a1583042-ovnkube-script-lib\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.940551    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-cni-bin\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941003    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-kubernetes\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941025    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e89102ba-1b0e-47d7-8bae-1f53a65984c6-tmp\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941049    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-var-lib-cni-multus\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941076    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-run-netns\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941086    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-run-ovn-kubernetes\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941092    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b256b7b-d193-4847-a3c7-c0868823bbae-cni-binary-copy\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.942053 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941103    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e962466-c4aa-4d88-b80a-1827a1583042-ovn-node-metrics-cert\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941089    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f11105f6-69d8-452b-9704-a4a7922fa418-konnectivity-ca\") pod \"konnectivity-agent-b5l6p\" (UID: \"f11105f6-69d8-452b-9704-a4a7922fa418\") " pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941138    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-device-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941171    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-sysctl-d\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941218    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-modprobe-d\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941237    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6-iptables-alerter-script\") pod \"iptables-alerter-lc2zw\" (UID: \"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6\") " pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941292    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-multus-socket-dir-parent\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941296    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-run-k8s-cni-cncf-io\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941318    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-cni-netd\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941346    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-registration-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941360    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-cnibin\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941376    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-etc-openvswitch\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941404    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xgtrt\" (UniqueName: \"kubernetes.io/projected/0e962466-c4aa-4d88-b80a-1827a1583042-kube-api-access-xgtrt\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941431    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941456    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-sys-fs\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941456    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-kubernetes\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941481    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-systemd\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.942849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941505    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-run-systemd\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941505    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-sys\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941524    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ae40396-0221-4852-a40a-ec38478dad13-multus-daemon-config\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941546    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f11105f6-69d8-452b-9704-a4a7922fa418-agent-certs\") pod \"konnectivity-agent-b5l6p\" (UID: \"f11105f6-69d8-452b-9704-a4a7922fa418\") " pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941551    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-run-openvswitch\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941585    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-sysconfig\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941622    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-device-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941644    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-lib-modules\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941685    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ccp5c\" (UniqueName: \"kubernetes.io/projected/e89102ba-1b0e-47d7-8bae-1f53a65984c6-kube-api-access-ccp5c\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941702    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0b256b7b-d193-4847-a3c7-c0868823bbae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941708    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941735    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-sysctl-d\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941776    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-system-cni-dir\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941806    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-os-release\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941831    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941860    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-kubelet\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941884    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0e962466-c4aa-4d88-b80a-1827a1583042-env-overrides\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.943610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941911    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-system-cni-dir\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941933    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf-serviceca\") pod \"node-ca-w8bpd\" (UID: \"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf\") " pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941953    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.941963    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-var-lib-cni-bin\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942008    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-var-lib-cni-multus\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942018    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-var-lib-openvswitch\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942030    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-sysconfig\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942058    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-system-cni-dir\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942074    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-os-release\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942089    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-os-release\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942100    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-sys-fs\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942115    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-kubelet\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942158    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-cni-netd\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942169    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-multus-socket-dir-parent\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942199    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942211    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-etc-openvswitch\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942229    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0b256b7b-d193-4847-a3c7-c0868823bbae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.944573 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942241    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-lib-modules\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942256    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6-iptables-alerter-script\") pod \"iptables-alerter-lc2zw\" (UID: \"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6\") " pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942259    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-systemd-units\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942283    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-os-release\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942164    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942306    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0e962466-c4aa-4d88-b80a-1827a1583042-ovnkube-config\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942335    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-run\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942359    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6-host-slash\") pod \"iptables-alerter-lc2zw\" (UID: \"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6\") " pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942377    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942418    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-run-multus-certs\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942428    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-sys\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942457    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-systemd\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942467    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2795d99a-b1c0-4151-b90a-0d6ce3e19488-registration-dir\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942501    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-systemd-units\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942538    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-host-run-netns\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942587    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e89102ba-1b0e-47d7-8bae-1f53a65984c6-run\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942585    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0e962466-c4aa-4d88-b80a-1827a1583042-ovnkube-script-lib\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942381    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-run-multus-certs\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.945398 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:32.942663    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942712    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6-host-slash\") pod \"iptables-alerter-lc2zw\" (UID: \"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6\") " pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942768    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xjv7m\" (UniqueName: \"kubernetes.io/projected/838a057a-e096-49a9-a086-9202ea786b05-kube-api-access-xjv7m\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942799    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-log-socket\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942823    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-node-log\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942845    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0b256b7b-d193-4847-a3c7-c0868823bbae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942875    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-system-cni-dir\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942895    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zxh7v\" (UniqueName: \"kubernetes.io/projected/2795d99a-b1c0-4151-b90a-0d6ce3e19488-kube-api-access-zxh7v\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942907    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-log-socket\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942921    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-multus-cni-dir\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942946    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0e962466-c4aa-4d88-b80a-1827a1583042-node-log\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.942983    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-run-netns\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:32.943062    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs podName:838a057a-e096-49a9-a086-9202ea786b05 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:33.443009982 +0000 UTC m=+3.126879299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs") pod "network-metrics-daemon-2grdq" (UID: "838a057a-e096-49a9-a086-9202ea786b05") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.943082    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-multus-conf-dir\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.943196    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-cnibin\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.943340    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b256b7b-d193-4847-a3c7-c0868823bbae-cnibin\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.943379    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-multus-conf-dir\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.946182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.943416    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-host-run-netns\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.946776 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.943454    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ae40396-0221-4852-a40a-ec38478dad13-multus-cni-dir\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.946776 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.944067    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0e962466-c4aa-4d88-b80a-1827a1583042-env-overrides\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.946776 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.944081    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0e962466-c4aa-4d88-b80a-1827a1583042-ovnkube-config\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.946776 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.945438    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e89102ba-1b0e-47d7-8bae-1f53a65984c6-etc-tuned\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.946776 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.945475    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e89102ba-1b0e-47d7-8bae-1f53a65984c6-tmp\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.946776 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.945539    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e962466-c4aa-4d88-b80a-1827a1583042-ovn-node-metrics-cert\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.946776 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.946335    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f11105f6-69d8-452b-9704-a4a7922fa418-agent-certs\") pod \"konnectivity-agent-b5l6p\" (UID: \"f11105f6-69d8-452b-9704-a4a7922fa418\") " pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:32.951945 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:32.951924    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 20 15:31:32.952115 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:32.952102    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 20 15:31:32.952208 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:32.952197    2566 projected.go:194] Error preparing data for projected volume kube-api-access-d95ml for pod openshift-network-diagnostics/network-check-target-mrqh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:32.952343 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:32.952333    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml podName:93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:33.452314078 +0000 UTC m=+3.136183398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d95ml" (UniqueName: "kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml") pod "network-check-target-mrqh2" (UID: "93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:32.952782 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.952737    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjv7m\" (UniqueName: \"kubernetes.io/projected/838a057a-e096-49a9-a086-9202ea786b05-kube-api-access-xjv7m\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:32.953118 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.953097    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgn7q\" (UniqueName: \"kubernetes.io/projected/0b256b7b-d193-4847-a3c7-c0868823bbae-kube-api-access-rgn7q\") pod \"multus-additional-cni-plugins-bhg2x\" (UID: \"0b256b7b-d193-4847-a3c7-c0868823bbae\") " pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:32.953281 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.953253    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxh7v\" (UniqueName: \"kubernetes.io/projected/2795d99a-b1c0-4151-b90a-0d6ce3e19488-kube-api-access-zxh7v\") pod \"aws-ebs-csi-driver-node-tsk65\" (UID: \"2795d99a-b1c0-4151-b90a-0d6ce3e19488\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:32.954631 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.954579    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccp5c\" (UniqueName: \"kubernetes.io/projected/e89102ba-1b0e-47d7-8bae-1f53a65984c6-kube-api-access-ccp5c\") pod \"tuned-hvk6w\" (UID: \"e89102ba-1b0e-47d7-8bae-1f53a65984c6\") " pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:32.954813 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.954794    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfzdn\" (UniqueName: \"kubernetes.io/projected/20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf-kube-api-access-bfzdn\") pod \"node-ca-w8bpd\" (UID: \"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf\") " pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:32.954976 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.954925    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgtrt\" (UniqueName: \"kubernetes.io/projected/0e962466-c4aa-4d88-b80a-1827a1583042-kube-api-access-xgtrt\") pod \"ovnkube-node-gj2r6\" (UID: \"0e962466-c4aa-4d88-b80a-1827a1583042\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:32.956124 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.956103    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnsqx\" (UniqueName: \"kubernetes.io/projected/7ae40396-0221-4852-a40a-ec38478dad13-kube-api-access-mnsqx\") pod \"multus-56bdg\" (UID: \"7ae40396-0221-4852-a40a-ec38478dad13\") " pod="openshift-multus/multus-56bdg"
May 20 15:31:32.956510 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:32.956489    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47ln4\" (UniqueName: \"kubernetes.io/projected/b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6-kube-api-access-47ln4\") pod \"iptables-alerter-lc2zw\" (UID: \"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6\") " pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:33.121189 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.121157    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:33.128131 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.128106    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:33.137717 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.137693    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65"
May 20 15:31:33.143371 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.143347    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hvk6w"
May 20 15:31:33.151107 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.151086    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-56bdg"
May 20 15:31:33.157660 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.157641    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lc2zw"
May 20 15:31:33.165073 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.165054    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w8bpd"
May 20 15:31:33.170591 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.170568    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bhg2x"
May 20 15:31:33.395844 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:33.395807    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e962466_c4aa_4d88_b80a_1827a1583042.slice/crio-1639bfa091026d120029bfd5ce556f33192c0365a6c3550a323373d7096e5191 WatchSource:0}: Error finding container 1639bfa091026d120029bfd5ce556f33192c0365a6c3550a323373d7096e5191: Status 404 returned error can't find the container with id 1639bfa091026d120029bfd5ce556f33192c0365a6c3550a323373d7096e5191
May 20 15:31:33.398153 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:31:33.398129    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode89102ba_1b0e_47d7_8bae_1f53a65984c6.slice/crio-d9fb82207082da90150547dfd22aa2226554b154cf28b8b40dc1adbf2895ceba WatchSource:0}: Error finding container d9fb82207082da90150547dfd22aa2226554b154cf28b8b40dc1adbf2895ceba: Status 404 returned error can't find the container with id d9fb82207082da90150547dfd22aa2226554b154cf28b8b40dc1adbf2895ceba
May 20 15:31:33.446886 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.446863    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:33.447001 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:33.446985    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:33.447046 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:33.447039    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs podName:838a057a-e096-49a9-a086-9202ea786b05 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:34.447025929 +0000 UTC m=+4.130895246 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs") pod "network-metrics-daemon-2grdq" (UID: "838a057a-e096-49a9-a086-9202ea786b05") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:33.547691 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.547659    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:33.547922 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:33.547843    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 20 15:31:33.547922 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:33.547869    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 20 15:31:33.547922 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:33.547880    2566 projected.go:194] Error preparing data for projected volume kube-api-access-d95ml for pod openshift-network-diagnostics/network-check-target-mrqh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:33.548113 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:33.547935    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml podName:93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:34.547915771 +0000 UTC m=+4.231785096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d95ml" (UniqueName: "kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml") pod "network-check-target-mrqh2" (UID: "93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:33.876218 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.876162    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-19 15:26:31 +0000 UTC" deadline="2027-12-21 08:14:19.658024684 +0000 UTC"
May 20 15:31:33.876218 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.876196    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13912h42m45.781832848s"
May 20 15:31:33.938963 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.938868    2566 generic.go:358] "Generic (PLEG): container finished" podID="b4fa92cd324f2b8adbc6bdb5146df37d" containerID="209f9f455d0c3210f5e447820f2d499020c30298c264c12325d204ea1f6c3db6" exitCode=0
May 20 15:31:33.938963 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.938933    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal" event={"ID":"b4fa92cd324f2b8adbc6bdb5146df37d","Type":"ContainerDied","Data":"209f9f455d0c3210f5e447820f2d499020c30298c264c12325d204ea1f6c3db6"}
May 20 15:31:33.970912 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.946232    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal" event={"ID":"49adea86e64f32c62411eb9c86b01363","Type":"ContainerStarted","Data":"1105bdd5eae79a6242a837524b6bf9d893a9618ff687dcf660780e2434c6436b"}
May 20 15:31:33.970912 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.949163    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lc2zw" event={"ID":"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6","Type":"ContainerStarted","Data":"6c90d8289427b1a0785697e21087bb8ebe3ed967962cb61e148e03b05aeb30ad"}
May 20 15:31:33.970912 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.955854    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65" event={"ID":"2795d99a-b1c0-4151-b90a-0d6ce3e19488","Type":"ContainerStarted","Data":"dfffda593db67bf14a90c2c6acd3c12b5286e36409969aa064ccdaaa2f20c21a"}
May 20 15:31:33.977347 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.977317    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hvk6w" event={"ID":"e89102ba-1b0e-47d7-8bae-1f53a65984c6","Type":"ContainerStarted","Data":"d9fb82207082da90150547dfd22aa2226554b154cf28b8b40dc1adbf2895ceba"}
May 20 15:31:33.982588 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.982563    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" event={"ID":"0b256b7b-d193-4847-a3c7-c0868823bbae","Type":"ContainerStarted","Data":"915dfac503b39ad1f19170efcfad29dfa64e7887a472807783b16988cc3922d9"}
May 20 15:31:33.992367 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.992337    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-56bdg" event={"ID":"7ae40396-0221-4852-a40a-ec38478dad13","Type":"ContainerStarted","Data":"f6abc560c485130aca7bcbc4839a4fc0bd297d5a25453db13b07a5ee940f9bf7"}
May 20 15:31:33.995083 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:33.995058    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w8bpd" event={"ID":"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf","Type":"ContainerStarted","Data":"f5671487ece00c063d1a95dd7a911415573a0a0466e452b5598f390b64ca4c90"}
May 20 15:31:34.007208 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:34.007170    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b5l6p" event={"ID":"f11105f6-69d8-452b-9704-a4a7922fa418","Type":"ContainerStarted","Data":"78aae591ea4de192e71cbdeada5940d047dc30f7b24a5cf94877f94961b6d91e"}
May 20 15:31:34.016851 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:34.016820    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerStarted","Data":"1639bfa091026d120029bfd5ce556f33192c0365a6c3550a323373d7096e5191"}
May 20 15:31:34.462107 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:34.462023    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:34.462264 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:34.462199    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:34.462316 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:34.462266    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs podName:838a057a-e096-49a9-a086-9202ea786b05 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:36.462246175 +0000 UTC m=+6.146115504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs") pod "network-metrics-daemon-2grdq" (UID: "838a057a-e096-49a9-a086-9202ea786b05") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:34.562946 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:34.562907    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:34.563163 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:34.563141    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 20 15:31:34.563241 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:34.563165    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 20 15:31:34.563241 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:34.563178    2566 projected.go:194] Error preparing data for projected volume kube-api-access-d95ml for pod openshift-network-diagnostics/network-check-target-mrqh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:34.563241 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:34.563240    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml podName:93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:36.563222122 +0000 UTC m=+6.247091446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d95ml" (UniqueName: "kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml") pod "network-check-target-mrqh2" (UID: "93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:34.929802 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:34.929712    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:34.930228 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:34.929850    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:34.930299 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:34.930280    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:34.930406 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:34.930384    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:35.025846 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:35.025805    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal" event={"ID":"b4fa92cd324f2b8adbc6bdb5146df37d","Type":"ContainerStarted","Data":"e85726d8d001de7ca5f0890807b4ec93d38e8908f984b3adeb0d48a017974b5a"}
May 20 15:31:35.039466 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:35.038728    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-16.ec2.internal" podStartSLOduration=3.038710558 podStartE2EDuration="3.038710558s" podCreationTimestamp="2026-05-20 15:31:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:31:33.980001939 +0000 UTC m=+3.663871279" watchObservedRunningTime="2026-05-20 15:31:35.038710558 +0000 UTC m=+4.722579897"
May 20 15:31:36.480838 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:36.480785    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:36.481276 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:36.480933    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:36.481276 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:36.480999    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs podName:838a057a-e096-49a9-a086-9202ea786b05 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:40.480980422 +0000 UTC m=+10.164849757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs") pod "network-metrics-daemon-2grdq" (UID: "838a057a-e096-49a9-a086-9202ea786b05") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:36.582582 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:36.581900    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:36.582582 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:36.582130    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 20 15:31:36.582582 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:36.582155    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 20 15:31:36.582582 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:36.582170    2566 projected.go:194] Error preparing data for projected volume kube-api-access-d95ml for pod openshift-network-diagnostics/network-check-target-mrqh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:36.582582 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:36.582235    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml podName:93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:40.582216663 +0000 UTC m=+10.266085983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d95ml" (UniqueName: "kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml") pod "network-check-target-mrqh2" (UID: "93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:36.928604 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:36.928564    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:36.928791 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:36.928691    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:36.929145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:36.929123    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:36.929259 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:36.929228    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:38.928508 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:38.927884    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:38.928508 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:38.928028    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:38.928508 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:38.928381    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:38.928508 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:38.928465    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:40.515486 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:40.515023    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:40.515486 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:40.515174    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:40.515486 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:40.515229    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs podName:838a057a-e096-49a9-a086-9202ea786b05 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:48.515215662 +0000 UTC m=+18.199084978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs") pod "network-metrics-daemon-2grdq" (UID: "838a057a-e096-49a9-a086-9202ea786b05") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:40.615454 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:40.615412    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:40.615636 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:40.615601    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 20 15:31:40.615636 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:40.615621    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 20 15:31:40.615636 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:40.615634    2566 projected.go:194] Error preparing data for projected volume kube-api-access-d95ml for pod openshift-network-diagnostics/network-check-target-mrqh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:40.615810 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:40.615691    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml podName:93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:48.615674484 +0000 UTC m=+18.299543803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d95ml" (UniqueName: "kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml") pod "network-check-target-mrqh2" (UID: "93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:40.930270 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:40.929997    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:40.930426 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:40.930315    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:40.930719 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:40.930686    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:40.930842 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:40.930807    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:42.075188 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.075124    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-16.ec2.internal" podStartSLOduration=10.075105501 podStartE2EDuration="10.075105501s" podCreationTimestamp="2026-05-20 15:31:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:31:35.039450837 +0000 UTC m=+4.723320176" watchObservedRunningTime="2026-05-20 15:31:42.075105501 +0000 UTC m=+11.758974843"
May 20 15:31:42.075877 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.075857    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-9lvfg"]
May 20 15:31:42.080911 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.080890    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.080999 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:42.080972    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:42.128791 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.128740    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df155f89-7ce7-4d42-8e53-867e99dd42f2-dbus\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.128953 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.128844    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df155f89-7ce7-4d42-8e53-867e99dd42f2-kubelet-config\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.128953 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.128909    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.230362 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.229858    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df155f89-7ce7-4d42-8e53-867e99dd42f2-dbus\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.230362 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.229903    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df155f89-7ce7-4d42-8e53-867e99dd42f2-kubelet-config\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.230362 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.229951    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.230362 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.229995    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df155f89-7ce7-4d42-8e53-867e99dd42f2-dbus\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.230362 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.230067    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df155f89-7ce7-4d42-8e53-867e99dd42f2-kubelet-config\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.230362 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:42.230103    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 20 15:31:42.230362 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:42.230159    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret podName:df155f89-7ce7-4d42-8e53-867e99dd42f2 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:42.730142596 +0000 UTC m=+12.414011914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret") pod "global-pull-secret-syncer-9lvfg" (UID: "df155f89-7ce7-4d42-8e53-867e99dd42f2") : object "kube-system"/"original-pull-secret" not registered
May 20 15:31:42.733167 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.733132    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:42.733342 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:42.733251    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 20 15:31:42.733342 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:42.733307    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret podName:df155f89-7ce7-4d42-8e53-867e99dd42f2 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:43.733294266 +0000 UTC m=+13.417163583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret") pod "global-pull-secret-syncer-9lvfg" (UID: "df155f89-7ce7-4d42-8e53-867e99dd42f2") : object "kube-system"/"original-pull-secret" not registered
May 20 15:31:42.930641 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.930617    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:42.930641 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:42.930650    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:42.930910 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:42.930723    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:42.930910 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:42.930880    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:43.741641 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:43.741550    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:43.742084 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:43.741728    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 20 15:31:43.742084 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:43.741810    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret podName:df155f89-7ce7-4d42-8e53-867e99dd42f2 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:45.741794678 +0000 UTC m=+15.425664001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret") pod "global-pull-secret-syncer-9lvfg" (UID: "df155f89-7ce7-4d42-8e53-867e99dd42f2") : object "kube-system"/"original-pull-secret" not registered
May 20 15:31:43.928536 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:43.928507    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:43.928716 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:43.928611    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:44.931166 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:44.931138    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:44.931631 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:44.931147    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:44.931631 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:44.931229    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:44.931631 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:44.931341    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:45.755416 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:45.755379    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:45.755574 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:45.755541    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 20 15:31:45.755634 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:45.755614    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret podName:df155f89-7ce7-4d42-8e53-867e99dd42f2 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:49.75559345 +0000 UTC m=+19.439462774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret") pod "global-pull-secret-syncer-9lvfg" (UID: "df155f89-7ce7-4d42-8e53-867e99dd42f2") : object "kube-system"/"original-pull-secret" not registered
May 20 15:31:45.928257 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:45.928225    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:45.928426 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:45.928336    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:46.930883 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:46.930851    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:46.931293 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:46.930897    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:46.931293 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:46.930983    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:46.931293 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:46.931137    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:47.927800 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:47.927764    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:47.928074 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:47.927891    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:48.578355 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:48.578313    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:48.578816 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:48.578482    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:48.578816 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:48.578562    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs podName:838a057a-e096-49a9-a086-9202ea786b05 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:04.578540669 +0000 UTC m=+34.262409989 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs") pod "network-metrics-daemon-2grdq" (UID: "838a057a-e096-49a9-a086-9202ea786b05") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:31:48.679276 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:48.679232    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:48.679470 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:48.679403    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 20 15:31:48.679470 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:48.679427    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 20 15:31:48.679470 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:48.679438    2566 projected.go:194] Error preparing data for projected volume kube-api-access-d95ml for pod openshift-network-diagnostics/network-check-target-mrqh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:48.679618 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:48.679506    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml podName:93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:04.679485634 +0000 UTC m=+34.363354955 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d95ml" (UniqueName: "kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml") pod "network-check-target-mrqh2" (UID: "93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:31:48.931151 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:48.931073    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:48.931151 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:48.931091    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:48.931351 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:48.931187    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:48.931351 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:48.931279    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:49.787466 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:49.787421    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:49.787897 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:49.787547    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 20 15:31:49.787897 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:49.787612    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret podName:df155f89-7ce7-4d42-8e53-867e99dd42f2 nodeName:}" failed. No retries permitted until 2026-05-20 15:31:57.787598715 +0000 UTC m=+27.471468036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret") pod "global-pull-secret-syncer-9lvfg" (UID: "df155f89-7ce7-4d42-8e53-867e99dd42f2") : object "kube-system"/"original-pull-secret" not registered
May 20 15:31:49.928640 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:49.928553    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:49.928808 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:49.928677    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:50.928508 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:50.928473    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:50.928881 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:50.928560    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:50.928881 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:50.928632    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:50.928881 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:50.928726    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:51.928469 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:51.928223    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:51.928615 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:51.928588    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:52.062553 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.059973    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65" event={"ID":"2795d99a-b1c0-4151-b90a-0d6ce3e19488","Type":"ContainerStarted","Data":"5e0dac9f16a907c73dfa66a4f8656ec3a4bcc6e482ce6618047f4b318d179b02"}
May 20 15:31:52.065281 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.065249    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hvk6w" event={"ID":"e89102ba-1b0e-47d7-8bae-1f53a65984c6","Type":"ContainerStarted","Data":"88fb81239d7b4ce87bfb6dce75eb622b0cae06df57c23ec359366b3b4f5958d1"}
May 20 15:31:52.068325 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.068066    2566 generic.go:358] "Generic (PLEG): container finished" podID="0b256b7b-d193-4847-a3c7-c0868823bbae" containerID="008a414b9b52665b588ea38beeb89755070a3ae8ad70e2c857205be56a4c5ae6" exitCode=0
May 20 15:31:52.068325 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.068146    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" event={"ID":"0b256b7b-d193-4847-a3c7-c0868823bbae","Type":"ContainerDied","Data":"008a414b9b52665b588ea38beeb89755070a3ae8ad70e2c857205be56a4c5ae6"}
May 20 15:31:52.071976 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.071939    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w8bpd" event={"ID":"20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf","Type":"ContainerStarted","Data":"2e6ee4fe3cadcd8879c8005d7a0526e7b0355fcd939d8ca3ebe9f229b285dae4"}
May 20 15:31:52.073519 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.073387    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b5l6p" event={"ID":"f11105f6-69d8-452b-9704-a4a7922fa418","Type":"ContainerStarted","Data":"c0e01b01083414f5fced5c821968abc962163d1d657fdaa6990a37f5c426e968"}
May 20 15:31:52.077737 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.076057    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:31:52.077737 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.077641    2566 generic.go:358] "Generic (PLEG): container finished" podID="0e962466-c4aa-4d88-b80a-1827a1583042" containerID="3b3f79d80ef04d45ac69223b3896d140cb772c65c3e7bcec0315cdad1f732628" exitCode=1
May 20 15:31:52.077737 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.077672    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerStarted","Data":"c2c1641c4b4586662a4e5805af1bdbf0790815d8f2e3f9007522726418383a13"}
May 20 15:31:52.077737 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.077689    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerDied","Data":"3b3f79d80ef04d45ac69223b3896d140cb772c65c3e7bcec0315cdad1f732628"}
May 20 15:31:52.077737 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.077702    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerStarted","Data":"7e3b82cd1d5e79800470e4cc67f6495ea922789edd3b9494ad39711a34f787fe"}
May 20 15:31:52.092839 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.092806    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hvk6w" podStartSLOduration=2.919671379 podStartE2EDuration="21.092796158s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="2026-05-20 15:31:33.399969351 +0000 UTC m=+3.083838668" lastFinishedPulling="2026-05-20 15:31:51.573094117 +0000 UTC m=+21.256963447" observedRunningTime="2026-05-20 15:31:52.092219266 +0000 UTC m=+21.776088606" watchObservedRunningTime="2026-05-20 15:31:52.092796158 +0000 UTC m=+21.776665496"
May 20 15:31:52.160320 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.160268    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b5l6p" podStartSLOduration=3.031706294 podStartE2EDuration="21.160247996s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="2026-05-20 15:31:33.402085021 +0000 UTC m=+3.085954340" lastFinishedPulling="2026-05-20 15:31:51.530626723 +0000 UTC m=+21.214496042" observedRunningTime="2026-05-20 15:31:52.159814475 +0000 UTC m=+21.843683815" watchObservedRunningTime="2026-05-20 15:31:52.160247996 +0000 UTC m=+21.844117337"
May 20 15:31:52.213162 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.213118    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w8bpd" podStartSLOduration=3.087406781 podStartE2EDuration="21.213102171s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="2026-05-20 15:31:33.404977763 +0000 UTC m=+3.088847081" lastFinishedPulling="2026-05-20 15:31:51.53067314 +0000 UTC m=+21.214542471" observedRunningTime="2026-05-20 15:31:52.212315353 +0000 UTC m=+21.896184692" watchObservedRunningTime="2026-05-20 15:31:52.213102171 +0000 UTC m=+21.896971512"
May 20 15:31:52.932403 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.932194    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:52.932870 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:52.932518    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:52.932947 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:52.932923    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:52.933043 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:52.933021    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:53.080141 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.080107    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-56bdg" event={"ID":"7ae40396-0221-4852-a40a-ec38478dad13","Type":"ContainerStarted","Data":"1cb6508e0519185c7d4e4d40ba906859cf13887bb5a17d3a0b455bfd94eedd5e"}
May 20 15:31:53.082486 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.082471    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:31:53.082791 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.082770    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerStarted","Data":"b7175f1a0c0fc115d081f16b8b8b28856e241a52eb1731d43ba408d102493b0f"}
May 20 15:31:53.082846 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.082801    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerStarted","Data":"814ea49b33433d5ad2c2101a97d437bce733b099424cccfad33ba50d52bb07ef"}
May 20 15:31:53.082846 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.082816    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerStarted","Data":"4d26a44a330300d86c2d9529cd511e5216f12420e4493d69038bc072fe3cc2ae"}
May 20 15:31:53.083975 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.083953    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lc2zw" event={"ID":"b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6","Type":"ContainerStarted","Data":"d778e730c061a77c14d629943df7b1d3fb9a5b62681720f8bd21db1ae3315506"}
May 20 15:31:53.085619 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.085587    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65" event={"ID":"2795d99a-b1c0-4151-b90a-0d6ce3e19488","Type":"ContainerStarted","Data":"62cc003d4945efe827476b471ab90bd70137f415abf1a55b04e801bbb4d2ba35"}
May 20 15:31:53.099815 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.099792    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 20 15:31:53.145457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.145359    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-56bdg" podStartSLOduration=3.6265712949999998 podStartE2EDuration="22.145344483s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="2026-05-20 15:31:33.407671262 +0000 UTC m=+3.091540581" lastFinishedPulling="2026-05-20 15:31:51.92644445 +0000 UTC m=+21.610313769" observedRunningTime="2026-05-20 15:31:53.120861538 +0000 UTC m=+22.804730877" watchObservedRunningTime="2026-05-20 15:31:53.145344483 +0000 UTC m=+22.829213857"
May 20 15:31:53.145630 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.145530    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lc2zw" podStartSLOduration=3.981491512 podStartE2EDuration="22.145522382s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="2026-05-20 15:31:33.410454597 +0000 UTC m=+3.094323922" lastFinishedPulling="2026-05-20 15:31:51.574485468 +0000 UTC m=+21.258354792" observedRunningTime="2026-05-20 15:31:53.144947395 +0000 UTC m=+22.828816734" watchObservedRunningTime="2026-05-20 15:31:53.145522382 +0000 UTC m=+22.829391722"
May 20 15:31:53.911734 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.911626    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-20T15:31:53.099812034Z","UUID":"f0313391-f0ca-4888-a9a3-93df42bc54eb","Handler":null,"Name":"","Endpoint":""}
May 20 15:31:53.915316 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.914859    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 20 15:31:53.915316 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.914901    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 20 15:31:53.927681 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:53.927651    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:53.927834 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:53.927795    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:54.928673 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:54.928641    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:54.929194 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:54.928789    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:54.929194 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:54.928838    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:54.929194 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:54.928930    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:55.090709 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:55.090677    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65" event={"ID":"2795d99a-b1c0-4151-b90a-0d6ce3e19488","Type":"ContainerStarted","Data":"9758bd49d10ae512fc8604f98757d0c53162ec388d60e8a1d6d8deaff6b8c925"}
May 20 15:31:55.093228 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:55.093200    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:31:55.093508 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:55.093488    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerStarted","Data":"19a4356985e6f13c5b50792bddfbe37a10647f2ac4d77db17bf7ce04870c82bd"}
May 20 15:31:55.109450 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:55.109398    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsk65" podStartSLOduration=3.415899273 podStartE2EDuration="24.109381267s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="2026-05-20 15:31:33.404673921 +0000 UTC m=+3.088543240" lastFinishedPulling="2026-05-20 15:31:54.098155914 +0000 UTC m=+23.782025234" observedRunningTime="2026-05-20 15:31:55.10863659 +0000 UTC m=+24.792505930" watchObservedRunningTime="2026-05-20 15:31:55.109381267 +0000 UTC m=+24.793250607"
May 20 15:31:55.838170 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:55.838131    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:55.838962 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:55.838940    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:31:55.927937 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:55.927903    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:55.928125 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:55.928031    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:56.928697 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:56.928187    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:56.928697 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:56.928191    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:56.928697 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:56.928326    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:56.928697 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:56.928396    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:57.850696 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:57.850476    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:57.850880 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:57.850610    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 20 15:31:57.850880 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:57.850780    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret podName:df155f89-7ce7-4d42-8e53-867e99dd42f2 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:13.850742071 +0000 UTC m=+43.534611388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret") pod "global-pull-secret-syncer-9lvfg" (UID: "df155f89-7ce7-4d42-8e53-867e99dd42f2") : object "kube-system"/"original-pull-secret" not registered
May 20 15:31:57.928498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:57.928463    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:57.928675 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:57.928589    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:58.100615 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.100582    2566 generic.go:358] "Generic (PLEG): container finished" podID="0b256b7b-d193-4847-a3c7-c0868823bbae" containerID="437b4b2f3449c1cbca1cb7a3fb352d5e227959299e986407127b26cee9a5c175" exitCode=0
May 20 15:31:58.101068 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.100667    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" event={"ID":"0b256b7b-d193-4847-a3c7-c0868823bbae","Type":"ContainerDied","Data":"437b4b2f3449c1cbca1cb7a3fb352d5e227959299e986407127b26cee9a5c175"}
May 20 15:31:58.103721 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.103705    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:31:58.104043 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.104010    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerStarted","Data":"1d0be503c95eed9ad05a293d85bca0524206f63ef607486ca0fe8c3ae04e3a66"}
May 20 15:31:58.104306 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.104292    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:58.104396 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.104309    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:58.104521 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.104506    2566 scope.go:117] "RemoveContainer" containerID="3b3f79d80ef04d45ac69223b3896d140cb772c65c3e7bcec0315cdad1f732628"
May 20 15:31:58.121174 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.121154    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:58.927879 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.927855    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:58.927997 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:58.927977    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:58.928104 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:58.928083    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:58.928228 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:58.928201    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:31:59.107762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.107522    2566 generic.go:358] "Generic (PLEG): container finished" podID="0b256b7b-d193-4847-a3c7-c0868823bbae" containerID="0ad418aa506b6b6fe15601fef27240937f5c1b0bc4c5d1a278c39995ac693b56" exitCode=0
May 20 15:31:59.107762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.107611    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" event={"ID":"0b256b7b-d193-4847-a3c7-c0868823bbae","Type":"ContainerDied","Data":"0ad418aa506b6b6fe15601fef27240937f5c1b0bc4c5d1a278c39995ac693b56"}
May 20 15:31:59.111140 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.111118    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:31:59.111533 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.111506    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" event={"ID":"0e962466-c4aa-4d88-b80a-1827a1583042","Type":"ContainerStarted","Data":"3901d6a75415e8dccc73e8dcbcfaecca962ceb15fb693702d21685226c073e6b"}
May 20 15:31:59.111854 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.111834    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:59.128176 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.128157    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:31:59.159394 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.159353    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6" podStartSLOduration=9.934246688 podStartE2EDuration="28.159340482s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="2026-05-20 15:31:33.398905892 +0000 UTC m=+3.082775209" lastFinishedPulling="2026-05-20 15:31:51.623999672 +0000 UTC m=+21.307869003" observedRunningTime="2026-05-20 15:31:59.158921793 +0000 UTC m=+28.842791131" watchObservedRunningTime="2026-05-20 15:31:59.159340482 +0000 UTC m=+28.843209820"
May 20 15:31:59.192458 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.192436    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mrqh2"]
May 20 15:31:59.192567 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.192539    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:31:59.192639 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:59.192622    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:31:59.192958 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.192938    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lvfg"]
May 20 15:31:59.193042 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.193032    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:31:59.193124 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:59.193109    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:31:59.196268 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.196245    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2grdq"]
May 20 15:31:59.196374 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:31:59.196351    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:31:59.196473 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:31:59.196455    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:32:00.115622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:00.115531    2566 generic.go:358] "Generic (PLEG): container finished" podID="0b256b7b-d193-4847-a3c7-c0868823bbae" containerID="f4ece2c64e3d243adb744d7f9ee7fcbe99eeb38dfd060ada6e8abb30f2af9c55" exitCode=0
May 20 15:32:00.115979 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:00.115617    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" event={"ID":"0b256b7b-d193-4847-a3c7-c0868823bbae","Type":"ContainerDied","Data":"f4ece2c64e3d243adb744d7f9ee7fcbe99eeb38dfd060ada6e8abb30f2af9c55"}
May 20 15:32:00.931499 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:00.931415    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:32:00.931683 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:00.931544    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:32:00.931683 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:00.931583    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:32:00.931683 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:00.931642    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:32:00.931683 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:00.931648    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:32:00.931910 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:00.931741    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:32:02.928286 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:02.928241    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:32:02.929279 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:02.928262    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:32:02.929279 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:02.928385    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrqh2" podUID="93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8"
May 20 15:32:02.929279 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:02.928397    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:32:02.929279 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:02.928481    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lvfg" podUID="df155f89-7ce7-4d42-8e53-867e99dd42f2"
May 20 15:32:02.929279 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:02.928585    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2grdq" podUID="838a057a-e096-49a9-a086-9202ea786b05"
May 20 15:32:02.985713 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:02.985681    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:32:02.985909 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:02.985862    2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 20 15:32:02.986356 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:02.986330    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b5l6p"
May 20 15:32:04.603469 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.603431    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:32:04.603955 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:04.603546    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:32:04.603955 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:04.603609    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs podName:838a057a-e096-49a9-a086-9202ea786b05 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:36.603593954 +0000 UTC m=+66.287463273 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs") pod "network-metrics-daemon-2grdq" (UID: "838a057a-e096-49a9-a086-9202ea786b05") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 20 15:32:04.643236 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.643170    2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-16.ec2.internal" event="NodeReady"
May 20 15:32:04.643383 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.643333    2566 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 20 15:32:04.704498 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.704459    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:32:04.704693 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:04.704613    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 20 15:32:04.704693 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:04.704638    2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 20 15:32:04.704693 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:04.704649    2566 projected.go:194] Error preparing data for projected volume kube-api-access-d95ml for pod openshift-network-diagnostics/network-check-target-mrqh2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:32:04.704856 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:04.704704    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml podName:93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:36.70469035 +0000 UTC m=+66.388559667 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d95ml" (UniqueName: "kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml") pod "network-check-target-mrqh2" (UID: "93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 20 15:32:04.721654 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.720854    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7cc776758b-h8gbp"]
May 20 15:32:04.726768 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.726722    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"]
May 20 15:32:04.726907 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.726886    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:04.730832 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.730813    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:04.731110 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.731093    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
May 20 15:32:04.731786 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.731766    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 20 15:32:04.734853 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.734836    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 20 15:32:04.735280 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.735248    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
May 20 15:32:04.738593 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.738571    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 20 15:32:04.738905 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.738888    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
May 20 15:32:04.739015 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.738958    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 20 15:32:04.739133 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.739120    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
May 20 15:32:04.739214 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.739168    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 20 15:32:04.739352 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.739336    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-62gbd\""
May 20 15:32:04.739407 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.739341    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tcxbp\""
May 20 15:32:04.748736 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.748716    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 20 15:32:04.758756 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.758722    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cc776758b-h8gbp"]
May 20 15:32:04.760025 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.759850    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-lnvrg"]
May 20 15:32:04.764252 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.764233    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"]
May 20 15:32:04.764375 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.764357    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:04.768063 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.767866    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"]
May 20 15:32:04.768234 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.768213    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.771908    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.771963    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.772211    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.772344    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.772380    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6lctj\""
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.772703    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.773001    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-sxf6h\""
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.773126    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.773418    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.773642    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz"]
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.774089    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:04.774570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.774326    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 20 15:32:04.780416 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.780398    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
May 20 15:32:04.780897 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.780874    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"]
May 20 15:32:04.780992 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.780898    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
May 20 15:32:04.781061 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.781044    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz"
May 20 15:32:04.781296 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.781273    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
May 20 15:32:04.781406 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.781388    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-q484d\""
May 20 15:32:04.782645 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.782623    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 20 15:32:04.783836 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.783812    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
May 20 15:32:04.783933 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.783903    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-d5829"]
May 20 15:32:04.784066 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.784048    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:04.784426 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.784409    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
May 20 15:32:04.784624 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.784611    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vdvjn\""
May 20 15:32:04.786924 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.786905    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn"]
May 20 15:32:04.787049 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.787035    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:04.789857 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.789835    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"]
May 20 15:32:04.789985 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.789969    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn"
May 20 15:32:04.790869 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.790851    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
May 20 15:32:04.793031 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.793014    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"]
May 20 15:32:04.793148 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.793133    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pf4ql"]
May 20 15:32:04.793359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.793344    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:04.796026 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.795994    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:04.797291 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.797272    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 20 15:32:04.797688 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.797444    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 20 15:32:04.797688 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.797510    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mz6fq\""
May 20 15:32:04.797688 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.797637    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
May 20 15:32:04.797688 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.797656    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 20 15:32:04.798464 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.798448    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 20 15:32:04.798600 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.798581    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 20 15:32:04.798669 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.798632    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 20 15:32:04.798727 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.798675    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 20 15:32:04.798727 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.798448    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6fb5p\""
May 20 15:32:04.798854 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.798633    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 20 15:32:04.799583 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.799566    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 20 15:32:04.799698 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.799684    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 20 15:32:04.802845 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.802824    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5s49c\""
May 20 15:32:04.802936 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.802873    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vksc5\""
May 20 15:32:04.802936 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.802829    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 20 15:32:04.802936 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.802897    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-zcqcs\""
May 20 15:32:04.803095 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.802996    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
May 20 15:32:04.803219 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.803199    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 20 15:32:04.803287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.803236    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 20 15:32:04.803484 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.803468    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
May 20 15:32:04.804293 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.804274    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"]
May 20 15:32:04.808345 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.808325    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
May 20 15:32:04.814641 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.814613    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-d5829"]
May 20 15:32:04.815881 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.815792    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-lnvrg"]
May 20 15:32:04.820467 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.820156    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz"]
May 20 15:32:04.820467 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.820205    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn"]
May 20 15:32:04.829385 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.829356    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"]
May 20 15:32:04.830724 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.830701    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"]
May 20 15:32:04.830931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.830910    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"]
May 20 15:32:04.876956 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.876928    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pf4ql"]
May 20 15:32:04.877738 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.877714    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-txm4g"]
May 20 15:32:04.881578 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.881554    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-txm4g"
May 20 15:32:04.885046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.885026    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 20 15:32:04.885729 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.885311    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 20 15:32:04.885729 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.885383    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 20 15:32:04.885729 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.885419    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 20 15:32:04.885729 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.885609    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zngvt\""
May 20 15:32:04.892151 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.892129    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-txm4g"]
May 20 15:32:04.905994 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.905927    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e1fffa1-441f-4301-a2c4-f5c9034af757-ca-trust-extracted\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:04.905994 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.905966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-trusted-ca\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:04.906126 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.905992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-default-certificate\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:04.906126 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906018    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8582d06-acb4-42cc-bbe9-0bfa222bb480-config\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:04.906126 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906045    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjph2\" (UniqueName: \"kubernetes.io/projected/055ee774-d1d8-4bd2-beee-c246461e5f65-kube-api-access-cjph2\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:04.906126 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906076    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kxk9\" (UniqueName: \"kubernetes.io/projected/12a104cf-a5bb-402e-8b9e-af91b6c3b1be-kube-api-access-5kxk9\") pod \"kube-storage-version-migrator-operator-565644f49-v4zqc\" (UID: \"12a104cf-a5bb-402e-8b9e-af91b6c3b1be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:04.906126 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906102    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:04.906338 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906129    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ee774-d1d8-4bd2-beee-c246461e5f65-service-ca-bundle\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:04.906338 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906158    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnb8s\" (UniqueName: \"kubernetes.io/projected/4f76d47b-62e4-458d-ba83-cadd74738ed7-kube-api-access-hnb8s\") pod \"service-ca-operator-57b5b68b8-6prpl\" (UID: \"4f76d47b-62e4-458d-ba83-cadd74738ed7\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:04.906338 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzgmh\" (UniqueName: \"kubernetes.io/projected/0fafb2a2-8548-4791-962c-d20781434da7-kube-api-access-mzgmh\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:04.906338 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906215    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5qhp\" (UniqueName: \"kubernetes.io/projected/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-kube-api-access-g5qhp\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:04.906338 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906243    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-certificates\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:04.906338 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:04.906338 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f76d47b-62e4-458d-ba83-cadd74738ed7-config\") pod \"service-ca-operator-57b5b68b8-6prpl\" (UID: \"4f76d47b-62e4-458d-ba83-cadd74738ed7\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:04.906338 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906315    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lf9n\" (UniqueName: \"kubernetes.io/projected/e8582d06-acb4-42cc-bbe9-0bfa222bb480-kube-api-access-5lf9n\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:04.906338 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906340    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-bound-sa-token\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:04.906667 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906364    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12a104cf-a5bb-402e-8b9e-af91b6c3b1be-config\") pod \"kube-storage-version-migrator-operator-565644f49-v4zqc\" (UID: \"12a104cf-a5bb-402e-8b9e-af91b6c3b1be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:04.906667 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906392    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ee774-d1d8-4bd2-beee-c246461e5f65-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:04.906667 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906454    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:04.906667 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906491    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8582d06-acb4-42cc-bbe9-0bfa222bb480-serving-cert\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:04.906667 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906532    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/055ee774-d1d8-4bd2-beee-c246461e5f65-snapshots\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:04.906667 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-installation-pull-secrets\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:04.906667 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906591    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9wdv\" (UniqueName: \"kubernetes.io/projected/cc694183-b361-413f-8726-4469b1c63ef8-kube-api-access-t9wdv\") pod \"network-check-source-7f44b6dd87-vp9sn\" (UID: \"cc694183-b361-413f-8726-4469b1c63ef8\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn"
May 20 15:32:04.906667 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906625    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjz9n\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-kube-api-access-sjz9n\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:04.906667 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906663    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906689    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwpr9\" (UniqueName: \"kubernetes.io/projected/46be50ac-17f2-41d2-9080-4e2140a62824-kube-api-access-jwpr9\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906715    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0fafb2a2-8548-4791-962c-d20781434da7-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906788    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-stats-auth\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906817    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxsxs\" (UniqueName: \"kubernetes.io/projected/6ae3568f-2165-4389-a819-c7fd0096d88f-kube-api-access-xxsxs\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8582d06-acb4-42cc-bbe9-0bfa222bb480-trusted-ca\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906906    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906933    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/055ee774-d1d8-4bd2-beee-c246461e5f65-tmp\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.906996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f76d47b-62e4-458d-ba83-cadd74738ed7-serving-cert\") pod \"service-ca-operator-57b5b68b8-6prpl\" (UID: \"4f76d47b-62e4-458d-ba83-cadd74738ed7\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:04.907027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.907020    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/055ee774-d1d8-4bd2-beee-c246461e5f65-serving-cert\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:04.907372 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.907045    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-image-registry-private-configuration\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:04.907372 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.907094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12a104cf-a5bb-402e-8b9e-af91b6c3b1be-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-v4zqc\" (UID: \"12a104cf-a5bb-402e-8b9e-af91b6c3b1be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:04.907372 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.907129    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjnpq\" (UniqueName: \"kubernetes.io/projected/3f890232-16f6-40ea-ae7f-65f33c7ec51f-kube-api-access-wjnpq\") pod \"volume-data-source-validator-5cf8c89d64-96pcz\" (UID: \"3f890232-16f6-40ea-ae7f-65f33c7ec51f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz"
May 20 15:32:04.927866 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.927841    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:32:04.927973 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.927958    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:32:04.928256 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.928238    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:32:04.934855 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.934835    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 20 15:32:04.934962 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.934934    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 20 15:32:04.935170 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.935156    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hm9j9\""
May 20 15:32:04.935677 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:04.935510    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9zz48\""
May 20 15:32:05.007808 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.007773    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jwpr9\" (UniqueName: \"kubernetes.io/projected/46be50ac-17f2-41d2-9080-4e2140a62824-kube-api-access-jwpr9\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:05.007808 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.007814    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0fafb2a2-8548-4791-962c-d20781434da7-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:05.008052 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.007843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bg9m\" (UniqueName: \"kubernetes.io/projected/78900876-05c6-47ea-bf8a-9ede201d5796-kube-api-access-2bg9m\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.008052 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.007893    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-stats-auth\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.008052 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.007922    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xxsxs\" (UniqueName: \"kubernetes.io/projected/6ae3568f-2165-4389-a819-c7fd0096d88f-kube-api-access-xxsxs\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:05.008052 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.007973    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8582d06-acb4-42cc-bbe9-0bfa222bb480-trusted-ca\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:05.008052 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.007999    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.008052 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008039    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008070    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008094    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/055ee774-d1d8-4bd2-beee-c246461e5f65-tmp\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008121    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f76d47b-62e4-458d-ba83-cadd74738ed7-serving-cert\") pod \"service-ca-operator-57b5b68b8-6prpl\" (UID: \"4f76d47b-62e4-458d-ba83-cadd74738ed7\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008145    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/055ee774-d1d8-4bd2-beee-c246461e5f65-serving-cert\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008180    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-image-registry-private-configuration\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008204    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12a104cf-a5bb-402e-8b9e-af91b6c3b1be-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-v4zqc\" (UID: \"12a104cf-a5bb-402e-8b9e-af91b6c3b1be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008234    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wjnpq\" (UniqueName: \"kubernetes.io/projected/3f890232-16f6-40ea-ae7f-65f33c7ec51f-kube-api-access-wjnpq\") pod \"volume-data-source-validator-5cf8c89d64-96pcz\" (UID: \"3f890232-16f6-40ea-ae7f-65f33c7ec51f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008260    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e1fffa1-441f-4301-a2c4-f5c9034af757-ca-trust-extracted\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008283    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-trusted-ca\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008306    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-default-certificate\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.008342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008332    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8582d06-acb4-42cc-bbe9-0bfa222bb480-config\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008359    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cjph2\" (UniqueName: \"kubernetes.io/projected/055ee774-d1d8-4bd2-beee-c246461e5f65-kube-api-access-cjph2\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008391    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5kxk9\" (UniqueName: \"kubernetes.io/projected/12a104cf-a5bb-402e-8b9e-af91b6c3b1be-kube-api-access-5kxk9\") pod \"kube-storage-version-migrator-operator-565644f49-v4zqc\" (UID: \"12a104cf-a5bb-402e-8b9e-af91b6c3b1be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008417    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008448    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ee774-d1d8-4bd2-beee-c246461e5f65-service-ca-bundle\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008481    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hnb8s\" (UniqueName: \"kubernetes.io/projected/4f76d47b-62e4-458d-ba83-cadd74738ed7-kube-api-access-hnb8s\") pod \"service-ca-operator-57b5b68b8-6prpl\" (UID: \"4f76d47b-62e4-458d-ba83-cadd74738ed7\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008509    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mzgmh\" (UniqueName: \"kubernetes.io/projected/0fafb2a2-8548-4791-962c-d20781434da7-kube-api-access-mzgmh\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008543    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g5qhp\" (UniqueName: \"kubernetes.io/projected/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-kube-api-access-g5qhp\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008580    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78900876-05c6-47ea-bf8a-9ede201d5796-tmp-dir\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008613    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-certificates\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008644    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008668    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f76d47b-62e4-458d-ba83-cadd74738ed7-config\") pod \"service-ca-operator-57b5b68b8-6prpl\" (UID: \"4f76d47b-62e4-458d-ba83-cadd74738ed7\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008705    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5lf9n\" (UniqueName: \"kubernetes.io/projected/e8582d06-acb4-42cc-bbe9-0bfa222bb480-kube-api-access-5lf9n\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008717    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0fafb2a2-8548-4791-962c-d20781434da7-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008734    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-bound-sa-token\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008789    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12a104cf-a5bb-402e-8b9e-af91b6c3b1be-config\") pod \"kube-storage-version-migrator-operator-565644f49-v4zqc\" (UID: \"12a104cf-a5bb-402e-8b9e-af91b6c3b1be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:05.008885 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.008823    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ee774-d1d8-4bd2-beee-c246461e5f65-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.009614 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.009021    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 20 15:32:05.009614 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.009093    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert podName:6ae3568f-2165-4389-a819-c7fd0096d88f nodeName:}" failed. No retries permitted until 2026-05-20 15:32:05.509072204 +0000 UTC m=+35.192941532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert") pod "ingress-canary-pf4ql" (UID: "6ae3568f-2165-4389-a819-c7fd0096d88f") : secret "canary-serving-cert" not found
May 20 15:32:05.009614 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.009494    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ee774-d1d8-4bd2-beee-c246461e5f65-service-ca-bundle\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.009788 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.009633    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ee774-d1d8-4bd2-beee-c246461e5f65-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.009788 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.009672    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/78900876-05c6-47ea-bf8a-9ede201d5796-config-volume\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.009895 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.009811    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 20 15:32:05.009895 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.009862    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls podName:46be50ac-17f2-41d2-9080-4e2140a62824 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:05.509845723 +0000 UTC m=+35.193715054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls") pod "cluster-samples-operator-697587599-rxckv" (UID: "46be50ac-17f2-41d2-9080-4e2140a62824") : secret "samples-operator-tls" not found
May 20 15:32:05.010043 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.010028    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 20 15:32:05.010104 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010037    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/055ee774-d1d8-4bd2-beee-c246461e5f65-tmp\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.010104 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.010082    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls podName:0fafb2a2-8548-4791-962c-d20781434da7 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:05.510064928 +0000 UTC m=+35.193934260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-c9fsp" (UID: "0fafb2a2-8548-4791-962c-d20781434da7") : secret "cluster-monitoring-operator-tls" not found
May 20 15:32:05.010212 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010105    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.010212 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010137    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8582d06-acb4-42cc-bbe9-0bfa222bb480-serving-cert\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:05.010212 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010163    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/055ee774-d1d8-4bd2-beee-c246461e5f65-snapshots\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.010212 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010180    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8582d06-acb4-42cc-bbe9-0bfa222bb480-trusted-ca\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:05.010212 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010192    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-installation-pull-secrets\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.010450 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010261    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-certificates\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.010450 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.010308    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 20 15:32:05.010450 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.010350    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:05.510336604 +0000 UTC m=+35.194205922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : secret "router-metrics-certs-default" not found
May 20 15:32:05.010450 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010372    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wdv\" (UniqueName: \"kubernetes.io/projected/cc694183-b361-413f-8726-4469b1c63ef8-kube-api-access-t9wdv\") pod \"network-check-source-7f44b6dd87-vp9sn\" (UID: \"cc694183-b361-413f-8726-4469b1c63ef8\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn"
May 20 15:32:05.010648 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010497    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sjz9n\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-kube-api-access-sjz9n\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.010793 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.010766    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12a104cf-a5bb-402e-8b9e-af91b6c3b1be-config\") pod \"kube-storage-version-migrator-operator-565644f49-v4zqc\" (UID: \"12a104cf-a5bb-402e-8b9e-af91b6c3b1be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.011036    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8582d06-acb4-42cc-bbe9-0bfa222bb480-config\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.011147    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.011158    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d6ddf9cc-hrh2c: secret "image-registry-tls" not found
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.011203    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls podName:4e1fffa1-441f-4301-a2c4-f5c9034af757 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:05.511190945 +0000 UTC m=+35.195060266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls") pod "image-registry-68d6ddf9cc-hrh2c" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757") : secret "image-registry-tls" not found
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.011260    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e1fffa1-441f-4301-a2c4-f5c9034af757-ca-trust-extracted\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.011693    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.011691    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-trusted-ca\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.011869    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:05.511854003 +0000 UTC m=+35.195723343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : configmap references non-existent config key: service-ca.crt
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.012058    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/055ee774-d1d8-4bd2-beee-c246461e5f65-snapshots\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.012576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.012531    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f76d47b-62e4-458d-ba83-cadd74738ed7-config\") pod \"service-ca-operator-57b5b68b8-6prpl\" (UID: \"4f76d47b-62e4-458d-ba83-cadd74738ed7\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:05.014956 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.014909    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-default-certificate\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.015352 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.015330    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f76d47b-62e4-458d-ba83-cadd74738ed7-serving-cert\") pod \"service-ca-operator-57b5b68b8-6prpl\" (UID: \"4f76d47b-62e4-458d-ba83-cadd74738ed7\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:05.015432 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.015382    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/055ee774-d1d8-4bd2-beee-c246461e5f65-serving-cert\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.015432 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.015420    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-installation-pull-secrets\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.015777 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.015738    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-image-registry-private-configuration\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.016141 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.016120    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-stats-auth\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.016333 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.016316    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8582d06-acb4-42cc-bbe9-0bfa222bb480-serving-cert\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:05.017172 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.017149    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12a104cf-a5bb-402e-8b9e-af91b6c3b1be-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-v4zqc\" (UID: \"12a104cf-a5bb-402e-8b9e-af91b6c3b1be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:05.028429 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.028387    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5qhp\" (UniqueName: \"kubernetes.io/projected/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-kube-api-access-g5qhp\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.028677 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.028655    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwpr9\" (UniqueName: \"kubernetes.io/projected/46be50ac-17f2-41d2-9080-4e2140a62824-kube-api-access-jwpr9\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:05.029102 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.029058    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-bound-sa-token\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.029596 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.029554    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kxk9\" (UniqueName: \"kubernetes.io/projected/12a104cf-a5bb-402e-8b9e-af91b6c3b1be-kube-api-access-5kxk9\") pod \"kube-storage-version-migrator-operator-565644f49-v4zqc\" (UID: \"12a104cf-a5bb-402e-8b9e-af91b6c3b1be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:05.030862 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.030840    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxsxs\" (UniqueName: \"kubernetes.io/projected/6ae3568f-2165-4389-a819-c7fd0096d88f-kube-api-access-xxsxs\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:05.032653 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.032632    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lf9n\" (UniqueName: \"kubernetes.io/projected/e8582d06-acb4-42cc-bbe9-0bfa222bb480-kube-api-access-5lf9n\") pod \"console-operator-76dff48d59-lnvrg\" (UID: \"e8582d06-acb4-42cc-bbe9-0bfa222bb480\") " pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:05.036140 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.036099    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9wdv\" (UniqueName: \"kubernetes.io/projected/cc694183-b361-413f-8726-4469b1c63ef8-kube-api-access-t9wdv\") pod \"network-check-source-7f44b6dd87-vp9sn\" (UID: \"cc694183-b361-413f-8726-4469b1c63ef8\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn"
May 20 15:32:05.036622 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.036600    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjz9n\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-kube-api-access-sjz9n\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.038569 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.038550    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzgmh\" (UniqueName: \"kubernetes.io/projected/0fafb2a2-8548-4791-962c-d20781434da7-kube-api-access-mzgmh\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:05.044611 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.044587    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjph2\" (UniqueName: \"kubernetes.io/projected/055ee774-d1d8-4bd2-beee-c246461e5f65-kube-api-access-cjph2\") pod \"insights-operator-78bdfb888c-d5829\" (UID: \"055ee774-d1d8-4bd2-beee-c246461e5f65\") " pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.045966 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.045943    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjnpq\" (UniqueName: \"kubernetes.io/projected/3f890232-16f6-40ea-ae7f-65f33c7ec51f-kube-api-access-wjnpq\") pod \"volume-data-source-validator-5cf8c89d64-96pcz\" (UID: \"3f890232-16f6-40ea-ae7f-65f33c7ec51f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz"
May 20 15:32:05.045966 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.045971    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnb8s\" (UniqueName: \"kubernetes.io/projected/4f76d47b-62e4-458d-ba83-cadd74738ed7-kube-api-access-hnb8s\") pod \"service-ca-operator-57b5b68b8-6prpl\" (UID: \"4f76d47b-62e4-458d-ba83-cadd74738ed7\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:05.077049 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.077020    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:05.086854 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.086826    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"
May 20 15:32:05.102633 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.102611    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz"
May 20 15:32:05.112617 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.112592    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2bg9m\" (UniqueName: \"kubernetes.io/projected/78900876-05c6-47ea-bf8a-9ede201d5796-kube-api-access-2bg9m\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.112721 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.112703    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.112830 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.112816    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78900876-05c6-47ea-bf8a-9ede201d5796-tmp-dir\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.112893 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.112855    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/78900876-05c6-47ea-bf8a-9ede201d5796-config-volume\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.113215 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.112982    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 20 15:32:05.113215 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.113053    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls podName:78900876-05c6-47ea-bf8a-9ede201d5796 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:05.613034755 +0000 UTC m=+35.296904093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls") pod "dns-default-txm4g" (UID: "78900876-05c6-47ea-bf8a-9ede201d5796") : secret "dns-default-metrics-tls" not found
May 20 15:32:05.113363 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.113344    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78900876-05c6-47ea-bf8a-9ede201d5796-tmp-dir\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.113485 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.113463    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/78900876-05c6-47ea-bf8a-9ede201d5796-config-volume\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.121308 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.121286    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-d5829"
May 20 15:32:05.133147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.133126    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bg9m\" (UniqueName: \"kubernetes.io/projected/78900876-05c6-47ea-bf8a-9ede201d5796-kube-api-access-2bg9m\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.133762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.133724    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn"
May 20 15:32:05.141363 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.141337    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"
May 20 15:32:05.353319 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.353280    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-wmdhm"]
May 20 15:32:05.376632 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.376604    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.381350 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.381327    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w9tn4\""
May 20 15:32:05.416367 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.416336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5e55c923-f4ed-4bef-94ef-87bfbb26bf1b-hosts-file\") pod \"node-resolver-wmdhm\" (UID: \"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b\") " pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.416496 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.416447    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e55c923-f4ed-4bef-94ef-87bfbb26bf1b-tmp-dir\") pod \"node-resolver-wmdhm\" (UID: \"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b\") " pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.416631 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.416614    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq54m\" (UniqueName: \"kubernetes.io/projected/5e55c923-f4ed-4bef-94ef-87bfbb26bf1b-kube-api-access-vq54m\") pod \"node-resolver-wmdhm\" (UID: \"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b\") " pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.518014 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.517977    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:05.518014 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518022    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vq54m\" (UniqueName: \"kubernetes.io/projected/5e55c923-f4ed-4bef-94ef-87bfbb26bf1b-kube-api-access-vq54m\") pod \"node-resolver-wmdhm\" (UID: \"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b\") " pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.518234 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518050    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.518234 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518089    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5e55c923-f4ed-4bef-94ef-87bfbb26bf1b-hosts-file\") pod \"node-resolver-wmdhm\" (UID: \"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b\") " pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.518234 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518120    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:05.518234 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518153    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:05.518234 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518155    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 20 15:32:05.518234 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518185    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e55c923-f4ed-4bef-94ef-87bfbb26bf1b-tmp-dir\") pod \"node-resolver-wmdhm\" (UID: \"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b\") " pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.518234 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518205    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 20 15:32:05.518234 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518214    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5e55c923-f4ed-4bef-94ef-87bfbb26bf1b-hosts-file\") pod \"node-resolver-wmdhm\" (UID: \"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b\") " pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.518234 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518225    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert podName:6ae3568f-2165-4389-a819-c7fd0096d88f nodeName:}" failed. No retries permitted until 2026-05-20 15:32:06.518205348 +0000 UTC m=+36.202074682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert") pod "ingress-canary-pf4ql" (UID: "6ae3568f-2165-4389-a819-c7fd0096d88f") : secret "canary-serving-cert" not found
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518300    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518311    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d6ddf9cc-hrh2c: secret "image-registry-tls" not found
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518329    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518345    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls podName:4e1fffa1-441f-4301-a2c4-f5c9034af757 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:06.518330191 +0000 UTC m=+36.202199527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls") pod "image-registry-68d6ddf9cc-hrh2c" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757") : secret "image-registry-tls" not found
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518377    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518390    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518436    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls podName:46be50ac-17f2-41d2-9080-4e2140a62824 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:06.518426155 +0000 UTC m=+36.202295472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls") pod "cluster-samples-operator-697587599-rxckv" (UID: "46be50ac-17f2-41d2-9080-4e2140a62824") : secret "samples-operator-tls" not found
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518450    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:06.518442307 +0000 UTC m=+36.202311627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : secret "router-metrics-certs-default" not found
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518484    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518484    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:06.518469307 +0000 UTC m=+36.202338624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : configmap references non-existent config key: service-ca.crt
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.518496    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e55c923-f4ed-4bef-94ef-87bfbb26bf1b-tmp-dir\") pod \"node-resolver-wmdhm\" (UID: \"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b\") " pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.518543 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.518521    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls podName:0fafb2a2-8548-4791-962c-d20781434da7 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:06.518510657 +0000 UTC m=+36.202379974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-c9fsp" (UID: "0fafb2a2-8548-4791-962c-d20781434da7") : secret "cluster-monitoring-operator-tls" not found
May 20 15:32:05.533597 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.533571    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq54m\" (UniqueName: \"kubernetes.io/projected/5e55c923-f4ed-4bef-94ef-87bfbb26bf1b-kube-api-access-vq54m\") pod \"node-resolver-wmdhm\" (UID: \"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b\") " pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:05.619965 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.619882    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:05.620467 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.620041    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 20 15:32:05.620467 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:05.620116    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls podName:78900876-05c6-47ea-bf8a-9ede201d5796 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:06.620094516 +0000 UTC m=+36.303963837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls") pod "dns-default-txm4g" (UID: "78900876-05c6-47ea-bf8a-9ede201d5796") : secret "dns-default-metrics-tls" not found
May 20 15:32:05.688127 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:05.687883    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wmdhm"
May 20 15:32:06.527836 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:06.527793    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:06.528030 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:06.527860    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:06.528030 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:06.527898    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:06.528030 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.527957    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 20 15:32:06.528030 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528023    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 20 15:32:06.528030 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528034    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:08.5280031 +0000 UTC m=+38.211872429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : secret "router-metrics-certs-default" not found
May 20 15:32:06.528224 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528040    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d6ddf9cc-hrh2c: secret "image-registry-tls" not found
May 20 15:32:06.528224 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528094    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls podName:4e1fffa1-441f-4301-a2c4-f5c9034af757 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:08.52807793 +0000 UTC m=+38.211947257 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls") pod "image-registry-68d6ddf9cc-hrh2c" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757") : secret "image-registry-tls" not found
May 20 15:32:06.528224 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528110    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:08.528103181 +0000 UTC m=+38.211972506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : configmap references non-existent config key: service-ca.crt
May 20 15:32:06.528224 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:06.528161    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:06.528224 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:06.528195    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:06.528445 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:06.528232    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:06.528445 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528296    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 20 15:32:06.528445 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528313    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 20 15:32:06.528445 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528321    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 20 15:32:06.528445 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528342    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls podName:46be50ac-17f2-41d2-9080-4e2140a62824 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:08.528329007 +0000 UTC m=+38.212198324 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls") pod "cluster-samples-operator-697587599-rxckv" (UID: "46be50ac-17f2-41d2-9080-4e2140a62824") : secret "samples-operator-tls" not found
May 20 15:32:06.528445 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528363    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert podName:6ae3568f-2165-4389-a819-c7fd0096d88f nodeName:}" failed. No retries permitted until 2026-05-20 15:32:08.528354583 +0000 UTC m=+38.212223913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert") pod "ingress-canary-pf4ql" (UID: "6ae3568f-2165-4389-a819-c7fd0096d88f") : secret "canary-serving-cert" not found
May 20 15:32:06.528445 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.528375    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls podName:0fafb2a2-8548-4791-962c-d20781434da7 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:08.528369414 +0000 UTC m=+38.212238731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-c9fsp" (UID: "0fafb2a2-8548-4791-962c-d20781434da7") : secret "cluster-monitoring-operator-tls" not found
May 20 15:32:06.629057 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:06.629023    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:06.629470 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.629137    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 20 15:32:06.629470 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:06.629218    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls podName:78900876-05c6-47ea-bf8a-9ede201d5796 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:08.629196426 +0000 UTC m=+38.313065754 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls") pod "dns-default-txm4g" (UID: "78900876-05c6-47ea-bf8a-9ede201d5796") : secret "dns-default-metrics-tls" not found
May 20 15:32:06.852843 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:06.852796    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e55c923_f4ed_4bef_94ef_87bfbb26bf1b.slice/crio-13d34f2686b3c380379f6419b0749cbf5b6abb079844729bcdb741e0acafc04a WatchSource:0}: Error finding container 13d34f2686b3c380379f6419b0749cbf5b6abb079844729bcdb741e0acafc04a: Status 404 returned error can't find the container with id 13d34f2686b3c380379f6419b0749cbf5b6abb079844729bcdb741e0acafc04a
May 20 15:32:07.108632 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.108586    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc"]
May 20 15:32:07.114915 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.113643    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-lnvrg"]
May 20 15:32:07.122295 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:07.122067    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8582d06_acb4_42cc_bbe9_0bfa222bb480.slice/crio-cabe76497bca75a4b9529ac0119b2442f57d27e5179d10d29347809a1f8723ee WatchSource:0}: Error finding container cabe76497bca75a4b9529ac0119b2442f57d27e5179d10d29347809a1f8723ee: Status 404 returned error can't find the container with id cabe76497bca75a4b9529ac0119b2442f57d27e5179d10d29347809a1f8723ee
May 20 15:32:07.128306 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.128277    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz"]
May 20 15:32:07.128397 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.128311    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-d5829"]
May 20 15:32:07.128670 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:07.128643    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod055ee774_d1d8_4bd2_beee_c246461e5f65.slice/crio-d0d38cd5eb57fce813750fe2d9a208b456cfb076556b0adba89a39f4b2ec9358 WatchSource:0}: Error finding container d0d38cd5eb57fce813750fe2d9a208b456cfb076556b0adba89a39f4b2ec9358: Status 404 returned error can't find the container with id d0d38cd5eb57fce813750fe2d9a208b456cfb076556b0adba89a39f4b2ec9358
May 20 15:32:07.131772 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:07.131244    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f890232_16f6_40ea_ae7f_65f33c7ec51f.slice/crio-222839b2c9e2cac9a5516a960a9b90a273317c4386ab125b1069d4e829d512ac WatchSource:0}: Error finding container 222839b2c9e2cac9a5516a960a9b90a273317c4386ab125b1069d4e829d512ac: Status 404 returned error can't find the container with id 222839b2c9e2cac9a5516a960a9b90a273317c4386ab125b1069d4e829d512ac
May 20 15:32:07.131772 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.131447    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wmdhm" event={"ID":"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b","Type":"ContainerStarted","Data":"e3c6d4e93b65e52d1818f3b76b1b69eed008f553ff798a1481cbee5ff819b89e"}
May 20 15:32:07.131772 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.131494    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wmdhm" event={"ID":"5e55c923-f4ed-4bef-94ef-87bfbb26bf1b","Type":"ContainerStarted","Data":"13d34f2686b3c380379f6419b0749cbf5b6abb079844729bcdb741e0acafc04a"}
May 20 15:32:07.133142 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.133037    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc" event={"ID":"12a104cf-a5bb-402e-8b9e-af91b6c3b1be","Type":"ContainerStarted","Data":"b813ea5536c05252f5dd337bd3c720ffb77b379c1fc30af9076bba04106c7a8a"}
May 20 15:32:07.134072 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.134032    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" event={"ID":"e8582d06-acb4-42cc-bbe9-0bfa222bb480","Type":"ContainerStarted","Data":"cabe76497bca75a4b9529ac0119b2442f57d27e5179d10d29347809a1f8723ee"}
May 20 15:32:07.136219 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.136196    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl"]
May 20 15:32:07.139706 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:07.139685    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f76d47b_62e4_458d_ba83_cadd74738ed7.slice/crio-291b20cca509af8a27fb23dbad143cbb6c5b1e75ac99f18092e2e3e0b6421264 WatchSource:0}: Error finding container 291b20cca509af8a27fb23dbad143cbb6c5b1e75ac99f18092e2e3e0b6421264: Status 404 returned error can't find the container with id 291b20cca509af8a27fb23dbad143cbb6c5b1e75ac99f18092e2e3e0b6421264
May 20 15:32:07.146921 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.146898    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn"]
May 20 15:32:07.149963 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:07.149937    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc694183_b361_413f_8726_4469b1c63ef8.slice/crio-331aadcd950b38ff8e1eaa79b942898200aeb07f0007912d40b49ea7df24b504 WatchSource:0}: Error finding container 331aadcd950b38ff8e1eaa79b942898200aeb07f0007912d40b49ea7df24b504: Status 404 returned error can't find the container with id 331aadcd950b38ff8e1eaa79b942898200aeb07f0007912d40b49ea7df24b504
May 20 15:32:07.152549 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:07.151873    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-wmdhm" podStartSLOduration=2.151857898 podStartE2EDuration="2.151857898s" podCreationTimestamp="2026-05-20 15:32:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:32:07.151300657 +0000 UTC m=+36.835169998" watchObservedRunningTime="2026-05-20 15:32:07.151857898 +0000 UTC m=+36.835727237"
May 20 15:32:08.138191 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.138141    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn" event={"ID":"cc694183-b361-413f-8726-4469b1c63ef8","Type":"ContainerStarted","Data":"331aadcd950b38ff8e1eaa79b942898200aeb07f0007912d40b49ea7df24b504"}
May 20 15:32:08.140124 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.140067    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl" event={"ID":"4f76d47b-62e4-458d-ba83-cadd74738ed7","Type":"ContainerStarted","Data":"291b20cca509af8a27fb23dbad143cbb6c5b1e75ac99f18092e2e3e0b6421264"}
May 20 15:32:08.143554 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.143530    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-d5829" event={"ID":"055ee774-d1d8-4bd2-beee-c246461e5f65","Type":"ContainerStarted","Data":"d0d38cd5eb57fce813750fe2d9a208b456cfb076556b0adba89a39f4b2ec9358"}
May 20 15:32:08.151577 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.151550    2566 generic.go:358] "Generic (PLEG): container finished" podID="0b256b7b-d193-4847-a3c7-c0868823bbae" containerID="4a8ae7a75f38a0df6d3794396f402d4a54004c502b55c511cab1786bd0aa9613" exitCode=0
May 20 15:32:08.151679 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.151635    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" event={"ID":"0b256b7b-d193-4847-a3c7-c0868823bbae","Type":"ContainerDied","Data":"4a8ae7a75f38a0df6d3794396f402d4a54004c502b55c511cab1786bd0aa9613"}
May 20 15:32:08.156766 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.156283    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz" event={"ID":"3f890232-16f6-40ea-ae7f-65f33c7ec51f","Type":"ContainerStarted","Data":"222839b2c9e2cac9a5516a960a9b90a273317c4386ab125b1069d4e829d512ac"}
May 20 15:32:08.551170 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.551096    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:08.551351 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.551261    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:08.551351 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.551294    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:08.551351 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.551337    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:08.551514 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.551382    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:08.551514 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.551425    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:08.551601 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.551564    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 20 15:32:08.551601 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.551581    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d6ddf9cc-hrh2c: secret "image-registry-tls" not found
May 20 15:32:08.551689 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.551642    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls podName:4e1fffa1-441f-4301-a2c4-f5c9034af757 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:12.551622307 +0000 UTC m=+42.235491627 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls") pod "image-registry-68d6ddf9cc-hrh2c" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757") : secret "image-registry-tls" not found
May 20 15:32:08.552149 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.552128    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:12.552111776 +0000 UTC m=+42.235981096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : configmap references non-existent config key: service-ca.crt
May 20 15:32:08.552279 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.552206    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 20 15:32:08.552279 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.552243    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls podName:46be50ac-17f2-41d2-9080-4e2140a62824 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:12.552231876 +0000 UTC m=+42.236101193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls") pod "cluster-samples-operator-697587599-rxckv" (UID: "46be50ac-17f2-41d2-9080-4e2140a62824") : secret "samples-operator-tls" not found
May 20 15:32:08.552400 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.552294    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 20 15:32:08.552400 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.552330    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls podName:0fafb2a2-8548-4791-962c-d20781434da7 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:12.552318084 +0000 UTC m=+42.236187406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-c9fsp" (UID: "0fafb2a2-8548-4791-962c-d20781434da7") : secret "cluster-monitoring-operator-tls" not found
May 20 15:32:08.552400 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.552393    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 20 15:32:08.552582 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.552422    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert podName:6ae3568f-2165-4389-a819-c7fd0096d88f nodeName:}" failed. No retries permitted until 2026-05-20 15:32:12.55241209 +0000 UTC m=+42.236281410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert") pod "ingress-canary-pf4ql" (UID: "6ae3568f-2165-4389-a819-c7fd0096d88f") : secret "canary-serving-cert" not found
May 20 15:32:08.552582 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.552478    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 20 15:32:08.552582 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.552508    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:12.552498888 +0000 UTC m=+42.236368208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : secret "router-metrics-certs-default" not found
May 20 15:32:08.652214 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:08.652172    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:08.653618 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.652438    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 20 15:32:08.653618 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:08.652521    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls podName:78900876-05c6-47ea-bf8a-9ede201d5796 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:12.652500412 +0000 UTC m=+42.336369729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls") pod "dns-default-txm4g" (UID: "78900876-05c6-47ea-bf8a-9ede201d5796") : secret "dns-default-metrics-tls" not found
May 20 15:32:09.162716 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:09.162678    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" event={"ID":"0b256b7b-d193-4847-a3c7-c0868823bbae","Type":"ContainerStarted","Data":"45cf1230e96d9663910667116c6ddfccf79cd20aebc26e41563c815463be5d3e"}
May 20 15:32:10.168370 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:10.168331    2566 generic.go:358] "Generic (PLEG): container finished" podID="0b256b7b-d193-4847-a3c7-c0868823bbae" containerID="45cf1230e96d9663910667116c6ddfccf79cd20aebc26e41563c815463be5d3e" exitCode=0
May 20 15:32:10.168786 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:10.168394    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" event={"ID":"0b256b7b-d193-4847-a3c7-c0868823bbae","Type":"ContainerDied","Data":"45cf1230e96d9663910667116c6ddfccf79cd20aebc26e41563c815463be5d3e"}
May 20 15:32:12.586811 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:12.586575    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:12.586841    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.586735    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.586959    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls podName:0fafb2a2-8548-4791-962c-d20781434da7 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:20.586936245 +0000 UTC m=+50.270805564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-c9fsp" (UID: "0fafb2a2-8548-4791-962c-d20781434da7") : secret "cluster-monitoring-operator-tls" not found
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.586962    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.586968    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:12.586879    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.587014    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:20.586998714 +0000 UTC m=+50.270868046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : secret "router-metrics-certs-default" not found
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.587071    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert podName:6ae3568f-2165-4389-a819-c7fd0096d88f nodeName:}" failed. No retries permitted until 2026-05-20 15:32:20.587032373 +0000 UTC m=+50.270901703 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert") pod "ingress-canary-pf4ql" (UID: "6ae3568f-2165-4389-a819-c7fd0096d88f") : secret "canary-serving-cert" not found
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:12.587158    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:12.587246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:12.587203    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:12.587648 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.587289    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 20 15:32:12.587648 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.587300    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d6ddf9cc-hrh2c: secret "image-registry-tls" not found
May 20 15:32:12.587648 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:12.587328    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:12.587648 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.587342    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls podName:4e1fffa1-441f-4301-a2c4-f5c9034af757 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:20.587328988 +0000 UTC m=+50.271198315 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls") pod "image-registry-68d6ddf9cc-hrh2c" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757") : secret "image-registry-tls" not found
May 20 15:32:12.587648 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.587422    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 20 15:32:12.587648 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.587439    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:20.587419969 +0000 UTC m=+50.271289299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : configmap references non-existent config key: service-ca.crt
May 20 15:32:12.587648 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.587475    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls podName:46be50ac-17f2-41d2-9080-4e2140a62824 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:20.587462658 +0000 UTC m=+50.271331981 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls") pod "cluster-samples-operator-697587599-rxckv" (UID: "46be50ac-17f2-41d2-9080-4e2140a62824") : secret "samples-operator-tls" not found
May 20 15:32:12.688596 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:12.688557    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:12.688791 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.688684    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 20 15:32:12.688791 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:12.688764    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls podName:78900876-05c6-47ea-bf8a-9ede201d5796 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:20.688734395 +0000 UTC m=+50.372603716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls") pod "dns-default-txm4g" (UID: "78900876-05c6-47ea-bf8a-9ede201d5796") : secret "dns-default-metrics-tls" not found
May 20 15:32:13.901331 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:13.901299    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:32:13.905594 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:13.905568    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df155f89-7ce7-4d42-8e53-867e99dd42f2-original-pull-secret\") pod \"global-pull-secret-syncer-9lvfg\" (UID: \"df155f89-7ce7-4d42-8e53-867e99dd42f2\") " pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:32:13.938217 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:13.938162    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lvfg"
May 20 15:32:14.092541 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.092493    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lvfg"]
May 20 15:32:14.096778 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:14.096735    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf155f89_7ce7_4d42_8e53_867e99dd42f2.slice/crio-ef0e358363de6328ec3e4d5741096a21edbcaa97ed943bf8ad25f1f18fe8d5ae WatchSource:0}: Error finding container ef0e358363de6328ec3e4d5741096a21edbcaa97ed943bf8ad25f1f18fe8d5ae: Status 404 returned error can't find the container with id ef0e358363de6328ec3e4d5741096a21edbcaa97ed943bf8ad25f1f18fe8d5ae
May 20 15:32:14.179288 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.179252    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc" event={"ID":"12a104cf-a5bb-402e-8b9e-af91b6c3b1be","Type":"ContainerStarted","Data":"c96c0414da09e0a9b9faed0ef964fefc40ad930fe23633b6ab5c1003a0e0ccdb"}
May 20 15:32:14.181406 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.181363    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-d5829" event={"ID":"055ee774-d1d8-4bd2-beee-c246461e5f65","Type":"ContainerStarted","Data":"f621a8d31e7dfee307a67d87a1dc84c024ba799db81a3e4e503e60684e9df731"}
May 20 15:32:14.184854 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.184828    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" event={"ID":"0b256b7b-d193-4847-a3c7-c0868823bbae","Type":"ContainerStarted","Data":"3a7e995efc209dd060b60649eb9141fd38cc7f124bdfaf6189e0d90ac5fb7fb7"}
May 20 15:32:14.194853 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.194791    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/0.log"
May 20 15:32:14.194853 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.194833    2566 generic.go:358] "Generic (PLEG): container finished" podID="e8582d06-acb4-42cc-bbe9-0bfa222bb480" containerID="4024befdc1cfd400c549999ca8b6fe65c632b5bae7d2f7d90d0a6f7583c73c8e" exitCode=255
May 20 15:32:14.194996 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.194869    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" event={"ID":"e8582d06-acb4-42cc-bbe9-0bfa222bb480","Type":"ContainerDied","Data":"4024befdc1cfd400c549999ca8b6fe65c632b5bae7d2f7d90d0a6f7583c73c8e"}
May 20 15:32:14.195386 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.195141    2566 scope.go:117] "RemoveContainer" containerID="4024befdc1cfd400c549999ca8b6fe65c632b5bae7d2f7d90d0a6f7583c73c8e"
May 20 15:32:14.197599 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.197577    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz" event={"ID":"3f890232-16f6-40ea-ae7f-65f33c7ec51f","Type":"ContainerStarted","Data":"fb36e0d91b5405b82ee2a92119b25079ee238cd380bb2056d343ea5b69d25a5c"}
May 20 15:32:14.199337 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.199317    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn" event={"ID":"cc694183-b361-413f-8726-4469b1c63ef8","Type":"ContainerStarted","Data":"7bdf2d66a862988f5fadf94c37a4d9b12c3115584fa1e3e56b43c267dc1c9022"}
May 20 15:32:14.200717 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.200698    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl" event={"ID":"4f76d47b-62e4-458d-ba83-cadd74738ed7","Type":"ContainerStarted","Data":"d2f3a55e98f940da9a8cacc4c8d34fef9de787dbf7590884dc2dc45364d776c3"}
May 20 15:32:14.201863 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.201846    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lvfg" event={"ID":"df155f89-7ce7-4d42-8e53-867e99dd42f2","Type":"ContainerStarted","Data":"ef0e358363de6328ec3e4d5741096a21edbcaa97ed943bf8ad25f1f18fe8d5ae"}
May 20 15:32:14.209455 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.209390    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc" podStartSLOduration=20.826087371 podStartE2EDuration="27.209380227s" podCreationTimestamp="2026-05-20 15:31:47 +0000 UTC" firstStartedPulling="2026-05-20 15:32:07.120812718 +0000 UTC m=+36.804682036" lastFinishedPulling="2026-05-20 15:32:13.50410556 +0000 UTC m=+43.187974892" observedRunningTime="2026-05-20 15:32:14.208619866 +0000 UTC m=+43.892489200" watchObservedRunningTime="2026-05-20 15:32:14.209380227 +0000 UTC m=+43.893249567"
May 20 15:32:14.275849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.275772    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-d5829" podStartSLOduration=20.915786349 podStartE2EDuration="27.275735815s" podCreationTimestamp="2026-05-20 15:31:47 +0000 UTC" firstStartedPulling="2026-05-20 15:32:07.139160813 +0000 UTC m=+36.823030130" lastFinishedPulling="2026-05-20 15:32:13.499110266 +0000 UTC m=+43.182979596" observedRunningTime="2026-05-20 15:32:14.247241115 +0000 UTC m=+43.931110456" watchObservedRunningTime="2026-05-20 15:32:14.275735815 +0000 UTC m=+43.959605161"
May 20 15:32:14.301690 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.301642    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-96pcz" podStartSLOduration=20.970705204 podStartE2EDuration="27.301623058s" podCreationTimestamp="2026-05-20 15:31:47 +0000 UTC" firstStartedPulling="2026-05-20 15:32:07.133336801 +0000 UTC m=+36.817206125" lastFinishedPulling="2026-05-20 15:32:13.464254649 +0000 UTC m=+43.148123979" observedRunningTime="2026-05-20 15:32:14.301263547 +0000 UTC m=+43.985132887" watchObservedRunningTime="2026-05-20 15:32:14.301623058 +0000 UTC m=+43.985492398"
May 20 15:32:14.338953 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.338902    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bhg2x" podStartSLOduration=9.633775596 podStartE2EDuration="43.338883338s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="2026-05-20 15:31:33.402554494 +0000 UTC m=+3.086423815" lastFinishedPulling="2026-05-20 15:32:07.107662233 +0000 UTC m=+36.791531557" observedRunningTime="2026-05-20 15:32:14.336562111 +0000 UTC m=+44.020431451" watchObservedRunningTime="2026-05-20 15:32:14.338883338 +0000 UTC m=+44.022752679"
May 20 15:32:14.356344 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.356279    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-vp9sn" podStartSLOduration=14.711095368 podStartE2EDuration="21.356261752s" podCreationTimestamp="2026-05-20 15:31:53 +0000 UTC" firstStartedPulling="2026-05-20 15:32:07.152618368 +0000 UTC m=+36.836487688" lastFinishedPulling="2026-05-20 15:32:13.79778474 +0000 UTC m=+43.481654072" observedRunningTime="2026-05-20 15:32:14.355250445 +0000 UTC m=+44.039119785" watchObservedRunningTime="2026-05-20 15:32:14.356261752 +0000 UTC m=+44.040131093"
May 20 15:32:14.380522 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:14.380473    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl" podStartSLOduration=21.025368911 podStartE2EDuration="27.380456126s" podCreationTimestamp="2026-05-20 15:31:47 +0000 UTC" firstStartedPulling="2026-05-20 15:32:07.143152735 +0000 UTC m=+36.827022053" lastFinishedPulling="2026-05-20 15:32:13.498239937 +0000 UTC m=+43.182109268" observedRunningTime="2026-05-20 15:32:14.38031744 +0000 UTC m=+44.064186780" watchObservedRunningTime="2026-05-20 15:32:14.380456126 +0000 UTC m=+44.064325460"
May 20 15:32:15.077988 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:15.077913    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:15.077988 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:15.077960    2566 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:15.207499 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:15.207469    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/1.log"
May 20 15:32:15.207887 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:15.207867    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/0.log"
May 20 15:32:15.207993 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:15.207906    2566 generic.go:358] "Generic (PLEG): container finished" podID="e8582d06-acb4-42cc-bbe9-0bfa222bb480" containerID="346e225010cd88d230a5bc80a2352509230e92db39e864ab0e98e7890ce3f2f3" exitCode=255
May 20 15:32:15.208678 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:15.208660    2566 scope.go:117] "RemoveContainer" containerID="346e225010cd88d230a5bc80a2352509230e92db39e864ab0e98e7890ce3f2f3"
May 20 15:32:15.208877 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:15.208858    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-lnvrg_openshift-console-operator(e8582d06-acb4-42cc-bbe9-0bfa222bb480)\"" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" podUID="e8582d06-acb4-42cc-bbe9-0bfa222bb480"
May 20 15:32:15.209114 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:15.209089    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" event={"ID":"e8582d06-acb4-42cc-bbe9-0bfa222bb480","Type":"ContainerDied","Data":"346e225010cd88d230a5bc80a2352509230e92db39e864ab0e98e7890ce3f2f3"}
May 20 15:32:15.211243 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:15.211223    2566 scope.go:117] "RemoveContainer" containerID="4024befdc1cfd400c549999ca8b6fe65c632b5bae7d2f7d90d0a6f7583c73c8e"
May 20 15:32:16.214631 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:16.214603    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/1.log"
May 20 15:32:16.215170 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:16.215052    2566 scope.go:117] "RemoveContainer" containerID="346e225010cd88d230a5bc80a2352509230e92db39e864ab0e98e7890ce3f2f3"
May 20 15:32:16.215285 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:16.215262    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-lnvrg_openshift-console-operator(e8582d06-acb4-42cc-bbe9-0bfa222bb480)\"" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" podUID="e8582d06-acb4-42cc-bbe9-0bfa222bb480"
May 20 15:32:17.218689 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:17.218656    2566 scope.go:117] "RemoveContainer" containerID="346e225010cd88d230a5bc80a2352509230e92db39e864ab0e98e7890ce3f2f3"
May 20 15:32:17.219151 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:17.218926    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-lnvrg_openshift-console-operator(e8582d06-acb4-42cc-bbe9-0bfa222bb480)\"" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" podUID="e8582d06-acb4-42cc-bbe9-0bfa222bb480"
May 20 15:32:17.605047 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:17.605012    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wmdhm_5e55c923-f4ed-4bef-94ef-87bfbb26bf1b/dns-node-resolver/0.log"
May 20 15:32:18.605315 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:18.605285    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w8bpd_20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf/node-ca/0.log"
May 20 15:32:19.224693 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:19.224653    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lvfg" event={"ID":"df155f89-7ce7-4d42-8e53-867e99dd42f2","Type":"ContainerStarted","Data":"a1545a8d50525ea52d4e49f61765fb09a657216b834c434937f8a46df1bdcf0c"}
May 20 15:32:19.248724 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:19.248674    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9lvfg" podStartSLOduration=32.918843927 podStartE2EDuration="37.248660435s" podCreationTimestamp="2026-05-20 15:31:42 +0000 UTC" firstStartedPulling="2026-05-20 15:32:14.098799722 +0000 UTC m=+43.782669039" lastFinishedPulling="2026-05-20 15:32:18.428616231 +0000 UTC m=+48.112485547" observedRunningTime="2026-05-20 15:32:19.248153504 +0000 UTC m=+48.932022844" watchObservedRunningTime="2026-05-20 15:32:19.248660435 +0000 UTC m=+48.932529774"
May 20 15:32:20.664519 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:20.664481    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:20.664519 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:20.664521    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:20.664555    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:20.664600    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664641    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:20.664652    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664655    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664721    2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664736    2566 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664776    2566 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68d6ddf9cc-hrh2c: secret "image-registry-tls" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664724    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls podName:0fafb2a2-8548-4791-962c-d20781434da7 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:36.664704169 +0000 UTC m=+66.348573504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-c9fsp" (UID: "0fafb2a2-8548-4791-962c-d20781434da7") : secret "cluster-monitoring-operator-tls" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664823    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls podName:46be50ac-17f2-41d2-9080-4e2140a62824 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:36.664810028 +0000 UTC m=+66.348679345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls") pod "cluster-samples-operator-697587599-rxckv" (UID: "46be50ac-17f2-41d2-9080-4e2140a62824") : secret "samples-operator-tls" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664741    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664837    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert podName:6ae3568f-2165-4389-a819-c7fd0096d88f nodeName:}" failed. No retries permitted until 2026-05-20 15:32:36.664828584 +0000 UTC m=+66.348697906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert") pod "ingress-canary-pf4ql" (UID: "6ae3568f-2165-4389-a819-c7fd0096d88f") : secret "canary-serving-cert" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:20.664859    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664905    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:36.66488348 +0000 UTC m=+66.348752797 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : secret "router-metrics-certs-default" not found
May 20 15:32:20.664976 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664927    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls podName:4e1fffa1-441f-4301-a2c4-f5c9034af757 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:36.664919553 +0000 UTC m=+66.348788871 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls") pod "image-registry-68d6ddf9cc-hrh2c" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757") : secret "image-registry-tls" not found
May 20 15:32:20.665438 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.664940    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle podName:36a5a1fd-37ff-4e52-9774-b0ab1c033b6d nodeName:}" failed. No retries permitted until 2026-05-20 15:32:36.66493397 +0000 UTC m=+66.348803286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle") pod "router-default-7cc776758b-h8gbp" (UID: "36a5a1fd-37ff-4e52-9774-b0ab1c033b6d") : configmap references non-existent config key: service-ca.crt
May 20 15:32:20.765392 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:20.765341    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:20.765574 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.765479    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 20 15:32:20.765574 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:20.765539    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls podName:78900876-05c6-47ea-bf8a-9ede201d5796 nodeName:}" failed. No retries permitted until 2026-05-20 15:32:36.765523148 +0000 UTC m=+66.449392465 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls") pod "dns-default-txm4g" (UID: "78900876-05c6-47ea-bf8a-9ede201d5796") : secret "dns-default-metrics-tls" not found
May 20 15:32:25.077926 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:25.077888    2566 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:25.077926 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:25.077935    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:25.078336 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:25.078290    2566 scope.go:117] "RemoveContainer" containerID="346e225010cd88d230a5bc80a2352509230e92db39e864ab0e98e7890ce3f2f3"
May 20 15:32:25.240439 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:25.240417    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:32:25.240809 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:25.240795    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/1.log"
May 20 15:32:25.240867 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:25.240829    2566 generic.go:358] "Generic (PLEG): container finished" podID="e8582d06-acb4-42cc-bbe9-0bfa222bb480" containerID="a4d03f9f61c89b45e74eadb19c82382974d08df6b460f66d787ec8eaa2e1ae48" exitCode=255
May 20 15:32:25.240902 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:25.240872    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" event={"ID":"e8582d06-acb4-42cc-bbe9-0bfa222bb480","Type":"ContainerDied","Data":"a4d03f9f61c89b45e74eadb19c82382974d08df6b460f66d787ec8eaa2e1ae48"}
May 20 15:32:25.240935 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:25.240904    2566 scope.go:117] "RemoveContainer" containerID="346e225010cd88d230a5bc80a2352509230e92db39e864ab0e98e7890ce3f2f3"
May 20 15:32:25.241184 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:25.241168    2566 scope.go:117] "RemoveContainer" containerID="a4d03f9f61c89b45e74eadb19c82382974d08df6b460f66d787ec8eaa2e1ae48"
May 20 15:32:25.241378 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:25.241357    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-lnvrg_openshift-console-operator(e8582d06-acb4-42cc-bbe9-0bfa222bb480)\"" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" podUID="e8582d06-acb4-42cc-bbe9-0bfa222bb480"
May 20 15:32:26.244932 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:26.244902    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:32:31.128934 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:31.128907    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj2r6"
May 20 15:32:35.077553 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:35.077501    2566 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:35.077553 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:35.077559    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:35.078030 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:35.077932    2566 scope.go:117] "RemoveContainer" containerID="a4d03f9f61c89b45e74eadb19c82382974d08df6b460f66d787ec8eaa2e1ae48"
May 20 15:32:35.078126 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:35.078108    2566 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-lnvrg_openshift-console-operator(e8582d06-acb4-42cc-bbe9-0bfa222bb480)\"" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" podUID="e8582d06-acb4-42cc-bbe9-0bfa222bb480"
May 20 15:32:36.604513 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.604469    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:32:36.607002 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.606982    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 20 15:32:36.617455 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.617433    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838a057a-e096-49a9-a086-9202ea786b05-metrics-certs\") pod \"network-metrics-daemon-2grdq\" (UID: \"838a057a-e096-49a9-a086-9202ea786b05\") " pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:32:36.705058 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.705025    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:36.705203 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.705067    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:36.705203 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.705107    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:32:36.705203 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.705147    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:36.705203 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.705177    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:36.705404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.705215    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:36.705404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.705255    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:36.705843 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.705817    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-service-ca-bundle\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:36.707828 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.707780    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ae3568f-2165-4389-a819-c7fd0096d88f-cert\") pod \"ingress-canary-pf4ql\" (UID: \"6ae3568f-2165-4389-a819-c7fd0096d88f\") " pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:36.707930 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.707882    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fafb2a2-8548-4791-962c-d20781434da7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-c9fsp\" (UID: \"0fafb2a2-8548-4791-962c-d20781434da7\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:36.708231 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.708214    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a5a1fd-37ff-4e52-9774-b0ab1c033b6d-metrics-certs\") pod \"router-default-7cc776758b-h8gbp\" (UID: \"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d\") " pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:36.708293 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.708271    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46be50ac-17f2-41d2-9080-4e2140a62824-samples-operator-tls\") pod \"cluster-samples-operator-697587599-rxckv\" (UID: \"46be50ac-17f2-41d2-9080-4e2140a62824\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:36.708332 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.708313    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"image-registry-68d6ddf9cc-hrh2c\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") " pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:36.708376 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.708362    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d95ml\" (UniqueName: \"kubernetes.io/projected/93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8-kube-api-access-d95ml\") pod \"network-check-target-mrqh2\" (UID: \"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8\") " pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:32:36.747557 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.747531    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hm9j9\""
May 20 15:32:36.752802 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.752783    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9zz48\""
May 20 15:32:36.755634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.755618    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2grdq"
May 20 15:32:36.761330 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.761309    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:32:36.805930 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.805896    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:36.809304 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.809026    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78900876-05c6-47ea-bf8a-9ede201d5796-metrics-tls\") pod \"dns-default-txm4g\" (UID: \"78900876-05c6-47ea-bf8a-9ede201d5796\") " pod="openshift-dns/dns-default-txm4g"
May 20 15:32:36.843856 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.843833    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tcxbp\""
May 20 15:32:36.851177 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.851156    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-62gbd\""
May 20 15:32:36.851403 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.851377    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:36.856855 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.856384    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:36.891996 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.891964    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2grdq"]
May 20 15:32:36.895885 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:36.895860    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod838a057a_e096_49a9_a086_9202ea786b05.slice/crio-a5edf435dd406109de8e169e8fdaf8de4c1bc9c84c21b81fdeea17f60988317b WatchSource:0}: Error finding container a5edf435dd406109de8e169e8fdaf8de4c1bc9c84c21b81fdeea17f60988317b: Status 404 returned error can't find the container with id a5edf435dd406109de8e169e8fdaf8de4c1bc9c84c21b81fdeea17f60988317b
May 20 15:32:36.897940 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.897732    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-q484d\""
May 20 15:32:36.905434 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.905413    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"
May 20 15:32:36.910663 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.910606    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mrqh2"]
May 20 15:32:36.915383 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.915164    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mz6fq\""
May 20 15:32:36.925177 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:36.920618    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ab2bf4_fc4f_4508_85f9_5b7ad38aa2e8.slice/crio-17c5618542d34879c5e9a2a105d29dac33f344b313ffa0f823646def2251e1fe WatchSource:0}: Error finding container 17c5618542d34879c5e9a2a105d29dac33f344b313ffa0f823646def2251e1fe: Status 404 returned error can't find the container with id 17c5618542d34879c5e9a2a105d29dac33f344b313ffa0f823646def2251e1fe
May 20 15:32:36.925177 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.922966    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"
May 20 15:32:36.953638 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.953259    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5s49c\""
May 20 15:32:36.960365 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.959297    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pf4ql"
May 20 15:32:36.999490 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:36.999193    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zngvt\""
May 20 15:32:37.007231 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.006800    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-txm4g"
May 20 15:32:37.028700 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.028642    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cc776758b-h8gbp"]
May 20 15:32:37.031313 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:37.031262    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a5a1fd_37ff_4e52_9774_b0ab1c033b6d.slice/crio-521fd2b47ad8b8c69e3ed454f7588fbc07fdd6f731df0edf4e04363968234cc0 WatchSource:0}: Error finding container 521fd2b47ad8b8c69e3ed454f7588fbc07fdd6f731df0edf4e04363968234cc0: Status 404 returned error can't find the container with id 521fd2b47ad8b8c69e3ed454f7588fbc07fdd6f731df0edf4e04363968234cc0
May 20 15:32:37.054283 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.054075    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"]
May 20 15:32:37.111938 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.110993    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv"]
May 20 15:32:37.146217 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.146168    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp"]
May 20 15:32:37.181035 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.180998    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pf4ql"]
May 20 15:32:37.184343 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:37.184317    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae3568f_2165_4389_a819_c7fd0096d88f.slice/crio-65f9444d91451316fa8a703bd761805d53d3a5cde6e33fb66e8eec7bfe214ab6 WatchSource:0}: Error finding container 65f9444d91451316fa8a703bd761805d53d3a5cde6e33fb66e8eec7bfe214ab6: Status 404 returned error can't find the container with id 65f9444d91451316fa8a703bd761805d53d3a5cde6e33fb66e8eec7bfe214ab6
May 20 15:32:37.200808 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.200601    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-txm4g"]
May 20 15:32:37.205145 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:37.205120    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78900876_05c6_47ea_bf8a_9ede201d5796.slice/crio-113b15b50d5dd10c3e4c32524db3a4bc7c8b1091571ff21f0bea7041076f1ca8 WatchSource:0}: Error finding container 113b15b50d5dd10c3e4c32524db3a4bc7c8b1091571ff21f0bea7041076f1ca8: Status 404 returned error can't find the container with id 113b15b50d5dd10c3e4c32524db3a4bc7c8b1091571ff21f0bea7041076f1ca8
May 20 15:32:37.276871 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.276835    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c" event={"ID":"4e1fffa1-441f-4301-a2c4-f5c9034af757","Type":"ContainerStarted","Data":"a88d351b518cabc8130a9e00ac82f9ae7c53a6b5b22e4688f4b5a6ae908cb299"}
May 20 15:32:37.276871 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.276881    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c" event={"ID":"4e1fffa1-441f-4301-a2c4-f5c9034af757","Type":"ContainerStarted","Data":"a800623257c572a2fecae70e842a9192be5efec60579910c1f3a0af9a14ec870"}
May 20 15:32:37.277085 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.277048    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:37.278484 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.278456    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cc776758b-h8gbp" event={"ID":"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d","Type":"ContainerStarted","Data":"0da01d0cb19f5cdb40776e4fa1fd45ba7b5fc4751e74a4f5620e9c080c1b0ac4"}
May 20 15:32:37.278620 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.278494    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cc776758b-h8gbp" event={"ID":"36a5a1fd-37ff-4e52-9774-b0ab1c033b6d","Type":"ContainerStarted","Data":"521fd2b47ad8b8c69e3ed454f7588fbc07fdd6f731df0edf4e04363968234cc0"}
May 20 15:32:37.279790 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.279735    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2grdq" event={"ID":"838a057a-e096-49a9-a086-9202ea786b05","Type":"ContainerStarted","Data":"a5edf435dd406109de8e169e8fdaf8de4c1bc9c84c21b81fdeea17f60988317b"}
May 20 15:32:37.280950 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.280925    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv" event={"ID":"46be50ac-17f2-41d2-9080-4e2140a62824","Type":"ContainerStarted","Data":"5a16efbec990643be9c2ebbbd93e4cab6db915cf95a0574c90bded1c7f41cf4e"}
May 20 15:32:37.282114 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.282088    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-txm4g" event={"ID":"78900876-05c6-47ea-bf8a-9ede201d5796","Type":"ContainerStarted","Data":"113b15b50d5dd10c3e4c32524db3a4bc7c8b1091571ff21f0bea7041076f1ca8"}
May 20 15:32:37.283133 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.283101    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pf4ql" event={"ID":"6ae3568f-2165-4389-a819-c7fd0096d88f","Type":"ContainerStarted","Data":"65f9444d91451316fa8a703bd761805d53d3a5cde6e33fb66e8eec7bfe214ab6"}
May 20 15:32:37.284356 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.284326    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp" event={"ID":"0fafb2a2-8548-4791-962c-d20781434da7","Type":"ContainerStarted","Data":"ba961cb6181c343a492a82cc9934c9f19d93c5173fde760d967563f9da30b543"}
May 20 15:32:37.285728 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.285710    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mrqh2" event={"ID":"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8","Type":"ContainerStarted","Data":"ce3f46e2e06c93175dd1c812a5ce610d0cbc25c0c189adf7401141713ebccfe3"}
May 20 15:32:37.285858 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.285731    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mrqh2" event={"ID":"93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8","Type":"ContainerStarted","Data":"17c5618542d34879c5e9a2a105d29dac33f344b313ffa0f823646def2251e1fe"}
May 20 15:32:37.285931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.285914    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:32:37.298852 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.298814    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c" podStartSLOduration=66.298802827 podStartE2EDuration="1m6.298802827s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:32:37.298436568 +0000 UTC m=+66.982305908" watchObservedRunningTime="2026-05-20 15:32:37.298802827 +0000 UTC m=+66.982672165"
May 20 15:32:37.313634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.313586    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mrqh2" podStartSLOduration=66.313574591 podStartE2EDuration="1m6.313574591s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:32:37.313007688 +0000 UTC m=+66.996877027" watchObservedRunningTime="2026-05-20 15:32:37.313574591 +0000 UTC m=+66.997443931"
May 20 15:32:37.333763 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.333692    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-7cc776758b-h8gbp" podStartSLOduration=50.333675156 podStartE2EDuration="50.333675156s" podCreationTimestamp="2026-05-20 15:31:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:32:37.333035686 +0000 UTC m=+67.016905025" watchObservedRunningTime="2026-05-20 15:32:37.333675156 +0000 UTC m=+67.017544497"
May 20 15:32:37.852421 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.852390    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:37.855399 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:37.855365    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:38.290629 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:38.290581    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:38.292507 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:38.292299    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7cc776758b-h8gbp"
May 20 15:32:39.293991 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:39.293901    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2grdq" event={"ID":"838a057a-e096-49a9-a086-9202ea786b05","Type":"ContainerStarted","Data":"c804d64271963309a7382bf027a2d64216a4b7c0d6a173c2263bc7ef9483b1c8"}
May 20 15:32:41.750512 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.750469    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"]
May 20 15:32:41.767561 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.767539    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"
May 20 15:32:41.775858 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.775806    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"]
May 20 15:32:41.780361 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.780321    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9gg5m\""
May 20 15:32:41.780525 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.780402    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 20 15:32:41.781144 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.780863    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 20 15:32:41.848861 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.848832    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2409784b-f7ea-431b-9ea5-9b1e5a2fccf4-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-mwcm6\" (UID: \"2409784b-f7ea-431b-9ea5-9b1e5a2fccf4\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"
May 20 15:32:41.849057 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.848866    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2409784b-f7ea-431b-9ea5-9b1e5a2fccf4-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mwcm6\" (UID: \"2409784b-f7ea-431b-9ea5-9b1e5a2fccf4\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"
May 20 15:32:41.851491 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.851454    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-75mp8"]
May 20 15:32:41.865530 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.865509    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:41.872512 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.872488    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b28xv\""
May 20 15:32:41.872837 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.872820    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 20 15:32:41.874596 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.874577    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 20 15:32:41.881164 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.881140    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-75mp8"]
May 20 15:32:41.950135 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.950100    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:41.950306 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.950169    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2409784b-f7ea-431b-9ea5-9b1e5a2fccf4-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-mwcm6\" (UID: \"2409784b-f7ea-431b-9ea5-9b1e5a2fccf4\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"
May 20 15:32:41.950306 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.950226    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2409784b-f7ea-431b-9ea5-9b1e5a2fccf4-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mwcm6\" (UID: \"2409784b-f7ea-431b-9ea5-9b1e5a2fccf4\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"
May 20 15:32:41.950306 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.950281    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-crio-socket\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:41.950459 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.950306    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-877ct\" (UniqueName: \"kubernetes.io/projected/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-kube-api-access-877ct\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:41.950459 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.950333    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-data-volume\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:41.950459 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.950401    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:41.950850 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.950823    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2409784b-f7ea-431b-9ea5-9b1e5a2fccf4-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-mwcm6\" (UID: \"2409784b-f7ea-431b-9ea5-9b1e5a2fccf4\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"
May 20 15:32:41.952903 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:41.952887    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2409784b-f7ea-431b-9ea5-9b1e5a2fccf4-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-mwcm6\" (UID: \"2409784b-f7ea-431b-9ea5-9b1e5a2fccf4\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"
May 20 15:32:42.051452 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.051422    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.051614 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.051473    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-crio-socket\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.051614 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.051492    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-877ct\" (UniqueName: \"kubernetes.io/projected/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-kube-api-access-877ct\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.051614 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.051511    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-data-volume\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.051614 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.051549    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.051614 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.051559    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-crio-socket\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.051973 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.051953    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-data-volume\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.052135 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.052120    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.053802 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.053783    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.069805 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.069778    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-877ct\" (UniqueName: \"kubernetes.io/projected/1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79-kube-api-access-877ct\") pod \"insights-runtime-extractor-75mp8\" (UID: \"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79\") " pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.078735 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.078716    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"
May 20 15:32:42.175119 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.175086    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-75mp8"
May 20 15:32:42.229263 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.229206    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6"]
May 20 15:32:42.303605 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.303573    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2grdq" event={"ID":"838a057a-e096-49a9-a086-9202ea786b05","Type":"ContainerStarted","Data":"32d38c85e9e880bb80f360b8af6393f1185f3e0246ff5de5bdbc4cc9497e0e97"}
May 20 15:32:42.305300 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.305274    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv" event={"ID":"46be50ac-17f2-41d2-9080-4e2140a62824","Type":"ContainerStarted","Data":"5f1b1bf4a397545297bd3ec2793558fe9393b8beb311fe5b5da4b2136113936f"}
May 20 15:32:42.305425 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.305304    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv" event={"ID":"46be50ac-17f2-41d2-9080-4e2140a62824","Type":"ContainerStarted","Data":"55f34558bdcdfa0271132a67cb173466ccb2d4bdf5acafaabbd303cc5873a6f4"}
May 20 15:32:42.307130 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.307096    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-txm4g" event={"ID":"78900876-05c6-47ea-bf8a-9ede201d5796","Type":"ContainerStarted","Data":"cd753c25dd98a1b29ef7eee105a8630caf49853f80c0144fae22b34365c10860"}
May 20 15:32:42.307229 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.307140    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-txm4g" event={"ID":"78900876-05c6-47ea-bf8a-9ede201d5796","Type":"ContainerStarted","Data":"20cff5369b46ace134395b4e1b336295022182cb21e39834034c78e218eed0f6"}
May 20 15:32:42.307392 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.307373    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-txm4g"
May 20 15:32:42.311549 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.311518    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pf4ql" event={"ID":"6ae3568f-2165-4389-a819-c7fd0096d88f","Type":"ContainerStarted","Data":"b185fba2d8491992713b28eb1530c9d33b4f086603b19994409ba74d974f69e4"}
May 20 15:32:42.313184 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.313153    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp" event={"ID":"0fafb2a2-8548-4791-962c-d20781434da7","Type":"ContainerStarted","Data":"de79227a7d39e9d9a769de2e6dcf65e00cbddb8ef297f877faf368dde7d8d810"}
May 20 15:32:42.314262 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.314241    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6" event={"ID":"2409784b-f7ea-431b-9ea5-9b1e5a2fccf4","Type":"ContainerStarted","Data":"3a365cafb6f7aafaa76c19f31f93f3ff77fa4825f43419ad74689c47188ffdb1"}
May 20 15:32:42.336132 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.336109    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-75mp8"]
May 20 15:32:42.339662 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:42.339634    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ccbb006_8c11_4e71_8ef0_8e4f2ab5bc79.slice/crio-963b809fb4bb5ad8e1eed941bfd4e8d9373941679fa0826c13c98c9018cbb777 WatchSource:0}: Error finding container 963b809fb4bb5ad8e1eed941bfd4e8d9373941679fa0826c13c98c9018cbb777: Status 404 returned error can't find the container with id 963b809fb4bb5ad8e1eed941bfd4e8d9373941679fa0826c13c98c9018cbb777
May 20 15:32:42.349801 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.349729    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2grdq" podStartSLOduration=69.763149752 podStartE2EDuration="1m11.349715994s" podCreationTimestamp="2026-05-20 15:31:31 +0000 UTC" firstStartedPulling="2026-05-20 15:32:36.898052343 +0000 UTC m=+66.581921662" lastFinishedPulling="2026-05-20 15:32:38.484618575 +0000 UTC m=+68.168487904" observedRunningTime="2026-05-20 15:32:42.348205079 +0000 UTC m=+72.032074419" watchObservedRunningTime="2026-05-20 15:32:42.349715994 +0000 UTC m=+72.033585333"
May 20 15:32:42.372227 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.372204    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"]
May 20 15:32:42.376802 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.376702    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-rxckv" podStartSLOduration=51.270334538 podStartE2EDuration="55.376686165s" podCreationTimestamp="2026-05-20 15:31:47 +0000 UTC" firstStartedPulling="2026-05-20 15:32:37.200203051 +0000 UTC m=+66.884072380" lastFinishedPulling="2026-05-20 15:32:41.306554677 +0000 UTC m=+70.990424007" observedRunningTime="2026-05-20 15:32:42.374605003 +0000 UTC m=+72.058474343" watchObservedRunningTime="2026-05-20 15:32:42.376686165 +0000 UTC m=+72.060555505"
May 20 15:32:42.395598 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.395575    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"]
May 20 15:32:42.395715 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.395682    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"
May 20 15:32:42.398787 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.398736    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
May 20 15:32:42.399096 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.399081    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wzwcz\""
May 20 15:32:42.416695 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.416656    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pf4ql" podStartSLOduration=34.299320443 podStartE2EDuration="38.416642968s" podCreationTimestamp="2026-05-20 15:32:04 +0000 UTC" firstStartedPulling="2026-05-20 15:32:37.186200988 +0000 UTC m=+66.870070319" lastFinishedPulling="2026-05-20 15:32:41.303523517 +0000 UTC m=+70.987392844" observedRunningTime="2026-05-20 15:32:42.415769633 +0000 UTC m=+72.099638970" watchObservedRunningTime="2026-05-20 15:32:42.416642968 +0000 UTC m=+72.100512307"
May 20 15:32:42.447717 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.447579    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-c9fsp" podStartSLOduration=51.298460588 podStartE2EDuration="55.447559583s" podCreationTimestamp="2026-05-20 15:31:47 +0000 UTC" firstStartedPulling="2026-05-20 15:32:37.154512003 +0000 UTC m=+66.838381319" lastFinishedPulling="2026-05-20 15:32:41.303610994 +0000 UTC m=+70.987480314" observedRunningTime="2026-05-20 15:32:42.447240207 +0000 UTC m=+72.131109565" watchObservedRunningTime="2026-05-20 15:32:42.447559583 +0000 UTC m=+72.131428923"
May 20 15:32:42.480279 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.480235    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-txm4g" podStartSLOduration=34.383969452 podStartE2EDuration="38.480219942s" podCreationTimestamp="2026-05-20 15:32:04 +0000 UTC" firstStartedPulling="2026-05-20 15:32:37.20721239 +0000 UTC m=+66.891081707" lastFinishedPulling="2026-05-20 15:32:41.303462866 +0000 UTC m=+70.987332197" observedRunningTime="2026-05-20 15:32:42.479329427 +0000 UTC m=+72.163198767" watchObservedRunningTime="2026-05-20 15:32:42.480219942 +0000 UTC m=+72.164089335"
May 20 15:32:42.554658 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.554628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/798c02f8-ccba-4c5e-895e-be114a5c569f-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf\" (UID: \"798c02f8-ccba-4c5e-895e-be114a5c569f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"
May 20 15:32:42.655137 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.655049    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/798c02f8-ccba-4c5e-895e-be114a5c569f-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf\" (UID: \"798c02f8-ccba-4c5e-895e-be114a5c569f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"
May 20 15:32:42.658691 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.658658    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/798c02f8-ccba-4c5e-895e-be114a5c569f-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf\" (UID: \"798c02f8-ccba-4c5e-895e-be114a5c569f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"
May 20 15:32:42.704792 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.704737    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"
May 20 15:32:42.846534 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:42.846503    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"]
May 20 15:32:42.849806 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:42.849772    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod798c02f8_ccba_4c5e_895e_be114a5c569f.slice/crio-9a3a53933251d7d21826487bc2738efcdd9dee0854248277e7898eeaa3797a31 WatchSource:0}: Error finding container 9a3a53933251d7d21826487bc2738efcdd9dee0854248277e7898eeaa3797a31: Status 404 returned error can't find the container with id 9a3a53933251d7d21826487bc2738efcdd9dee0854248277e7898eeaa3797a31
May 20 15:32:43.318021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:43.317984    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf" event={"ID":"798c02f8-ccba-4c5e-895e-be114a5c569f","Type":"ContainerStarted","Data":"9a3a53933251d7d21826487bc2738efcdd9dee0854248277e7898eeaa3797a31"}
May 20 15:32:43.319599 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:43.319553    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-75mp8" event={"ID":"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79","Type":"ContainerStarted","Data":"a290c9368860db0a7e1d4d301027d11659aa6b75b32feeedd1a8b933d7e22498"}
May 20 15:32:43.319599 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:43.319597    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-75mp8" event={"ID":"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79","Type":"ContainerStarted","Data":"963b809fb4bb5ad8e1eed941bfd4e8d9373941679fa0826c13c98c9018cbb777"}
May 20 15:32:44.324206 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:44.324172    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6" event={"ID":"2409784b-f7ea-431b-9ea5-9b1e5a2fccf4","Type":"ContainerStarted","Data":"eb6aec6d5bced6934c90d107d0cb1e5ced6844c34c11bea8d6b671363eb78d6a"}
May 20 15:32:44.325809 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:44.325725    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-75mp8" event={"ID":"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79","Type":"ContainerStarted","Data":"8dab8669dc69e6e52abb8e09c86581a85d452fafd4fb181c57ba626bfa753e3c"}
May 20 15:32:45.329872 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:45.329828    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf" event={"ID":"798c02f8-ccba-4c5e-895e-be114a5c569f","Type":"ContainerStarted","Data":"5b478d7d8c98aa15df5ebcc676a38eb9021c682667ce681a4a61b67494a40fde"}
May 20 15:32:45.330280 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:45.330173    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"
May 20 15:32:45.335198 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:45.335170    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf"
May 20 15:32:45.347715 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:45.347670    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf" podStartSLOduration=1.662701738 podStartE2EDuration="3.347658225s" podCreationTimestamp="2026-05-20 15:32:42 +0000 UTC" firstStartedPulling="2026-05-20 15:32:42.851982292 +0000 UTC m=+72.535851609" lastFinishedPulling="2026-05-20 15:32:44.536938769 +0000 UTC m=+74.220808096" observedRunningTime="2026-05-20 15:32:45.34623169 +0000 UTC m=+75.030101029" watchObservedRunningTime="2026-05-20 15:32:45.347658225 +0000 UTC m=+75.031527564"
May 20 15:32:45.363607 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:45.363569    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-mwcm6" podStartSLOduration=2.406291882 podStartE2EDuration="4.363560513s" podCreationTimestamp="2026-05-20 15:32:41 +0000 UTC" firstStartedPulling="2026-05-20 15:32:42.233997899 +0000 UTC m=+71.917867230" lastFinishedPulling="2026-05-20 15:32:44.191266531 +0000 UTC m=+73.875135861" observedRunningTime="2026-05-20 15:32:45.362093844 +0000 UTC m=+75.045963183" watchObservedRunningTime="2026-05-20 15:32:45.363560513 +0000 UTC m=+75.047429852"
May 20 15:32:45.928853 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:45.928822    2566 scope.go:117] "RemoveContainer" containerID="a4d03f9f61c89b45e74eadb19c82382974d08df6b460f66d787ec8eaa2e1ae48"
May 20 15:32:46.335433 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:46.335393    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-75mp8" event={"ID":"1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79","Type":"ContainerStarted","Data":"a2ce6506da55e00537a3d98699b3b9b2c35201833d2150b1935941743dac29ac"}
May 20 15:32:46.337122 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:46.337102    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:32:46.337242 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:46.337223    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" event={"ID":"e8582d06-acb4-42cc-bbe9-0bfa222bb480","Type":"ContainerStarted","Data":"23ca7c39b45006dd1b8b6ab583f9c9c8de49fa7dc77fc3b756b1c67663f4a26a"}
May 20 15:32:46.337502 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:46.337489    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:46.368483 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:46.368438    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-75mp8" podStartSLOduration=1.830907543 podStartE2EDuration="5.368426318s" podCreationTimestamp="2026-05-20 15:32:41 +0000 UTC" firstStartedPulling="2026-05-20 15:32:42.420499942 +0000 UTC m=+72.104369258" lastFinishedPulling="2026-05-20 15:32:45.958018702 +0000 UTC m=+75.641888033" observedRunningTime="2026-05-20 15:32:46.367308419 +0000 UTC m=+76.051177792" watchObservedRunningTime="2026-05-20 15:32:46.368426318 +0000 UTC m=+76.052295656"
May 20 15:32:46.404430 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:46.404384    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg" podStartSLOduration=53.030733125 podStartE2EDuration="59.404375524s" podCreationTimestamp="2026-05-20 15:31:47 +0000 UTC" firstStartedPulling="2026-05-20 15:32:07.124349029 +0000 UTC m=+36.808218346" lastFinishedPulling="2026-05-20 15:32:13.497991418 +0000 UTC m=+43.181860745" observedRunningTime="2026-05-20 15:32:46.404054299 +0000 UTC m=+76.087923639" watchObservedRunningTime="2026-05-20 15:32:46.404375524 +0000 UTC m=+76.088244862"
May 20 15:32:46.475287 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:46.475261    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-lnvrg"
May 20 15:32:50.861736 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:50.861698    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bxw26"]
May 20 15:32:50.864330 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:50.864310    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:50.866876 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:50.866855    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 20 15:32:50.866965 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:50.866856    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-n8vs4\""
May 20 15:32:50.867116 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:50.867098    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 20 15:32:50.867190 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:50.867137    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 20 15:32:50.868120 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:50.868104    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 20 15:32:51.020507 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.020477    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76b66068-e932-4815-8041-1c30a059518f-sys\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.020507 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.020513    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76b66068-e932-4815-8041-1c30a059518f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.020762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.020535    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-884b9\" (UniqueName: \"kubernetes.io/projected/76b66068-e932-4815-8041-1c30a059518f-kube-api-access-884b9\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.020762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.020557    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/76b66068-e932-4815-8041-1c30a059518f-node-exporter-textfile\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.020762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.020610    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76b66068-e932-4815-8041-1c30a059518f-metrics-client-ca\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.020762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.020709    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76b66068-e932-4815-8041-1c30a059518f-node-exporter-tls\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.020762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.020740    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/76b66068-e932-4815-8041-1c30a059518f-node-exporter-wtmp\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.020917 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.020822    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/76b66068-e932-4815-8041-1c30a059518f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.020917 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.020844    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/76b66068-e932-4815-8041-1c30a059518f-root\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.121947    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76b66068-e932-4815-8041-1c30a059518f-metrics-client-ca\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122021 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122007    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76b66068-e932-4815-8041-1c30a059518f-node-exporter-tls\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122194 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122034    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/76b66068-e932-4815-8041-1c30a059518f-node-exporter-wtmp\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122194 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122081    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/76b66068-e932-4815-8041-1c30a059518f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122194 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122108    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/76b66068-e932-4815-8041-1c30a059518f-root\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122194 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122154    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76b66068-e932-4815-8041-1c30a059518f-sys\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122194 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122157    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/76b66068-e932-4815-8041-1c30a059518f-root\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122194 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122178    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76b66068-e932-4815-8041-1c30a059518f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122194 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:51.122108    2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 20 15:32:51.122535 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122211    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/76b66068-e932-4815-8041-1c30a059518f-node-exporter-wtmp\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122535 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122220    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76b66068-e932-4815-8041-1c30a059518f-sys\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122535 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122214    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-884b9\" (UniqueName: \"kubernetes.io/projected/76b66068-e932-4815-8041-1c30a059518f-kube-api-access-884b9\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122535 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:32:51.122277    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b66068-e932-4815-8041-1c30a059518f-node-exporter-tls podName:76b66068-e932-4815-8041-1c30a059518f nodeName:}" failed. No retries permitted until 2026-05-20 15:32:51.622252887 +0000 UTC m=+81.306122212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/76b66068-e932-4815-8041-1c30a059518f-node-exporter-tls") pod "node-exporter-bxw26" (UID: "76b66068-e932-4815-8041-1c30a059518f") : secret "node-exporter-tls" not found
May 20 15:32:51.122535 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122296    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/76b66068-e932-4815-8041-1c30a059518f-node-exporter-textfile\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122808 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122565    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/76b66068-e932-4815-8041-1c30a059518f-node-exporter-textfile\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122808 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122654    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76b66068-e932-4815-8041-1c30a059518f-metrics-client-ca\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.122808 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.122684    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/76b66068-e932-4815-8041-1c30a059518f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.124520 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.124501    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76b66068-e932-4815-8041-1c30a059518f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.147645 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.147621    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-884b9\" (UniqueName: \"kubernetes.io/projected/76b66068-e932-4815-8041-1c30a059518f-kube-api-access-884b9\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.627179 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.627133    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76b66068-e932-4815-8041-1c30a059518f-node-exporter-tls\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.629609 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.629582    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/76b66068-e932-4815-8041-1c30a059518f-node-exporter-tls\") pod \"node-exporter-bxw26\" (UID: \"76b66068-e932-4815-8041-1c30a059518f\") " pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.773435 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:51.773406    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bxw26"
May 20 15:32:51.783019 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:51.782987    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b66068_e932_4815_8041_1c30a059518f.slice/crio-5fd97ce536adb30926873829d81484d0eb300a307b0411a6d04a554233df6205 WatchSource:0}: Error finding container 5fd97ce536adb30926873829d81484d0eb300a307b0411a6d04a554233df6205: Status 404 returned error can't find the container with id 5fd97ce536adb30926873829d81484d0eb300a307b0411a6d04a554233df6205
May 20 15:32:52.322246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:52.322206    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-txm4g"
May 20 15:32:52.358046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:52.357989    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bxw26" event={"ID":"76b66068-e932-4815-8041-1c30a059518f","Type":"ContainerStarted","Data":"5fd97ce536adb30926873829d81484d0eb300a307b0411a6d04a554233df6205"}
May 20 15:32:53.363105 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:53.363071    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bxw26" event={"ID":"76b66068-e932-4815-8041-1c30a059518f","Type":"ContainerStarted","Data":"643be5b220bcfed544912e9b2b5199bae9a48a682846e80ed9c1aa4ed192e29a"}
May 20 15:32:54.367331 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:54.367290    2566 generic.go:358] "Generic (PLEG): container finished" podID="76b66068-e932-4815-8041-1c30a059518f" containerID="643be5b220bcfed544912e9b2b5199bae9a48a682846e80ed9c1aa4ed192e29a" exitCode=0
May 20 15:32:54.367712 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:54.367370    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bxw26" event={"ID":"76b66068-e932-4815-8041-1c30a059518f","Type":"ContainerDied","Data":"643be5b220bcfed544912e9b2b5199bae9a48a682846e80ed9c1aa4ed192e29a"}
May 20 15:32:55.371571 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:55.371534    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bxw26" event={"ID":"76b66068-e932-4815-8041-1c30a059518f","Type":"ContainerStarted","Data":"b32c675e0de4800529cb248e8fa31611f020304697ee79aac68e38ba6954d760"}
May 20 15:32:55.371571 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:55.371569    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bxw26" event={"ID":"76b66068-e932-4815-8041-1c30a059518f","Type":"ContainerStarted","Data":"46f1558aebcc3ae716c6446324e5f90a9d3d0c50f788cca3de5c8d27bbc9ab02"}
May 20 15:32:55.391716 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:55.391664    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bxw26" podStartSLOduration=4.029424989 podStartE2EDuration="5.391651439s" podCreationTimestamp="2026-05-20 15:32:50 +0000 UTC" firstStartedPulling="2026-05-20 15:32:51.784857625 +0000 UTC m=+81.468726942" lastFinishedPulling="2026-05-20 15:32:53.147084068 +0000 UTC m=+82.830953392" observedRunningTime="2026-05-20 15:32:55.390060457 +0000 UTC m=+85.073929796" watchObservedRunningTime="2026-05-20 15:32:55.391651439 +0000 UTC m=+85.075520778"
May 20 15:32:57.162363 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.162326    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 20 15:32:57.165013 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.164994    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.168497 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.168468    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 20 15:32:57.168706 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.168690    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9g2kd\""
May 20 15:32:57.168822 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.168791    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 20 15:32:57.168822 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.168795    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 20 15:32:57.168953 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.168816    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 20 15:32:57.168953 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.168855    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 20 15:32:57.169036 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.169003    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 20 15:32:57.169076 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.169005    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 20 15:32:57.169235 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.169215    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 20 15:32:57.169812 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.169795    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2j4elu8qe5nrl\""
May 20 15:32:57.169904 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.169797    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 20 15:32:57.170046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.170031    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 20 15:32:57.170138 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.170047    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 20 15:32:57.170138 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.170059    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 20 15:32:57.171562 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.171542    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 20 15:32:57.179314 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.179290    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 20 15:32:57.273029 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.272994    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273029 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273032    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273293 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273064    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273293 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273082    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273293 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273155    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273293 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273194    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273293 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273219    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273293 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273246    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7cwv\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-kube-api-access-r7cwv\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273293    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273328    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273357    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273391    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273426    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273456    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273498    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.273576 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273562    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.274082 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273600    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.274082 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.273618    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.374841 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.374798    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.374863    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.374883    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.374908    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.374928    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.374950    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.374987    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375015    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375046 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375047    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375075    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375137    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375161    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375219    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375252    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375279    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375304    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375330    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375457 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375360    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r7cwv\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-kube-api-access-r7cwv\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.375953 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.375802    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.376446 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.376416    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.376555 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.376518    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.376619 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.376575    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.378426 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.378395    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.378794 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.378736    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.379008 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.378991    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.379127 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.379012    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.379455 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.379427    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.379701 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.379677    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.380312 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.380283    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.381030 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.381001    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.381124 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.381072    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.381627 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.381605    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.381926 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.381908    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.382038 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.382022    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.382191 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.382169    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.384027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.384003    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7cwv\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-kube-api-access-r7cwv\") pod \"prometheus-k8s-0\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.474659 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.474571    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:32:57.616302 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:57.616263    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 20 15:32:57.617574 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:32:57.617546    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94994f58_7484_4cf7_a2a6_70bab8b6f3e4.slice/crio-8fc603a1196c57e9fa1cf0868a9f1952a4a4b45ca26adb6ba71d2a9a1f57e68c WatchSource:0}: Error finding container 8fc603a1196c57e9fa1cf0868a9f1952a4a4b45ca26adb6ba71d2a9a1f57e68c: Status 404 returned error can't find the container with id 8fc603a1196c57e9fa1cf0868a9f1952a4a4b45ca26adb6ba71d2a9a1f57e68c
May 20 15:32:58.295014 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:58.294982    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:32:58.382769 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:58.382713    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerStarted","Data":"8fc603a1196c57e9fa1cf0868a9f1952a4a4b45ca26adb6ba71d2a9a1f57e68c"}
May 20 15:32:59.387010 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:59.386968    2566 generic.go:358] "Generic (PLEG): container finished" podID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerID="998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51" exitCode=0
May 20 15:32:59.387394 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:32:59.387061    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerDied","Data":"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"}
May 20 15:33:02.399150 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:02.399116    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerStarted","Data":"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"}
May 20 15:33:02.399150 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:02.399151    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerStarted","Data":"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"}
May 20 15:33:04.407767 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:04.407709    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerStarted","Data":"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"}
May 20 15:33:04.407767 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:04.407743    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerStarted","Data":"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"}
May 20 15:33:04.407767 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:04.407775    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerStarted","Data":"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"}
May 20 15:33:04.408344 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:04.407783    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerStarted","Data":"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"}
May 20 15:33:04.440610 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:04.440546    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.415298693 podStartE2EDuration="7.44052898s" podCreationTimestamp="2026-05-20 15:32:57 +0000 UTC" firstStartedPulling="2026-05-20 15:32:57.619533986 +0000 UTC m=+87.303403303" lastFinishedPulling="2026-05-20 15:33:03.644764273 +0000 UTC m=+93.328633590" observedRunningTime="2026-05-20 15:33:04.438926886 +0000 UTC m=+94.122796226" watchObservedRunningTime="2026-05-20 15:33:04.44052898 +0000 UTC m=+94.124398319"
May 20 15:33:05.186243 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:05.186212    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"]
May 20 15:33:07.474932 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:07.474893    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:33:08.291904 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:08.291867    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mrqh2"
May 20 15:33:20.454954 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:20.454919    2566 generic.go:358] "Generic (PLEG): container finished" podID="4f76d47b-62e4-458d-ba83-cadd74738ed7" containerID="d2f3a55e98f940da9a8cacc4c8d34fef9de787dbf7590884dc2dc45364d776c3" exitCode=0
May 20 15:33:20.455350 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:20.454993    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl" event={"ID":"4f76d47b-62e4-458d-ba83-cadd74738ed7","Type":"ContainerDied","Data":"d2f3a55e98f940da9a8cacc4c8d34fef9de787dbf7590884dc2dc45364d776c3"}
May 20 15:33:20.455350 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:20.455291    2566 scope.go:117] "RemoveContainer" containerID="d2f3a55e98f940da9a8cacc4c8d34fef9de787dbf7590884dc2dc45364d776c3"
May 20 15:33:21.459623 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:21.459576    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-6prpl" event={"ID":"4f76d47b-62e4-458d-ba83-cadd74738ed7","Type":"ContainerStarted","Data":"e047c9133369954ef0298fe7f4483293bccda712cdbd71a371bca2fff7dd701f"}
May 20 15:33:30.205849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.205777    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c" podUID="4e1fffa1-441f-4301-a2c4-f5c9034af757" containerName="registry" containerID="cri-o://a88d351b518cabc8130a9e00ac82f9ae7c53a6b5b22e4688f4b5a6ae908cb299" gracePeriod=30
May 20 15:33:30.485358 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.485297    2566 generic.go:358] "Generic (PLEG): container finished" podID="4e1fffa1-441f-4301-a2c4-f5c9034af757" containerID="a88d351b518cabc8130a9e00ac82f9ae7c53a6b5b22e4688f4b5a6ae908cb299" exitCode=0
May 20 15:33:30.485518 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.485381    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c" event={"ID":"4e1fffa1-441f-4301-a2c4-f5c9034af757","Type":"ContainerDied","Data":"a88d351b518cabc8130a9e00ac82f9ae7c53a6b5b22e4688f4b5a6ae908cb299"}
May 20 15:33:30.485518 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.485426    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c" event={"ID":"4e1fffa1-441f-4301-a2c4-f5c9034af757","Type":"ContainerDied","Data":"a800623257c572a2fecae70e842a9192be5efec60579910c1f3a0af9a14ec870"}
May 20 15:33:30.485518 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.485441    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a800623257c572a2fecae70e842a9192be5efec60579910c1f3a0af9a14ec870"
May 20 15:33:30.486814 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.486793    2566 generic.go:358] "Generic (PLEG): container finished" podID="055ee774-d1d8-4bd2-beee-c246461e5f65" containerID="f621a8d31e7dfee307a67d87a1dc84c024ba799db81a3e4e503e60684e9df731" exitCode=0
May 20 15:33:30.486918 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.486854    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-d5829" event={"ID":"055ee774-d1d8-4bd2-beee-c246461e5f65","Type":"ContainerDied","Data":"f621a8d31e7dfee307a67d87a1dc84c024ba799db81a3e4e503e60684e9df731"}
May 20 15:33:30.487194 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.487178    2566 scope.go:117] "RemoveContainer" containerID="f621a8d31e7dfee307a67d87a1dc84c024ba799db81a3e4e503e60684e9df731"
May 20 15:33:30.500020 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.499999    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:33:30.667469 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.667431    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-certificates\") pod \"4e1fffa1-441f-4301-a2c4-f5c9034af757\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") "
May 20 15:33:30.667469 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.667470    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") pod \"4e1fffa1-441f-4301-a2c4-f5c9034af757\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") "
May 20 15:33:30.667715 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.667522    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-trusted-ca\") pod \"4e1fffa1-441f-4301-a2c4-f5c9034af757\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") "
May 20 15:33:30.667715 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.667548    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-bound-sa-token\") pod \"4e1fffa1-441f-4301-a2c4-f5c9034af757\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") "
May 20 15:33:30.667715 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.667584    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e1fffa1-441f-4301-a2c4-f5c9034af757-ca-trust-extracted\") pod \"4e1fffa1-441f-4301-a2c4-f5c9034af757\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") "
May 20 15:33:30.667715 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.667610    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjz9n\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-kube-api-access-sjz9n\") pod \"4e1fffa1-441f-4301-a2c4-f5c9034af757\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") "
May 20 15:33:30.667715 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.667641    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-image-registry-private-configuration\") pod \"4e1fffa1-441f-4301-a2c4-f5c9034af757\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") "
May 20 15:33:30.667715 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.667689    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-installation-pull-secrets\") pod \"4e1fffa1-441f-4301-a2c4-f5c9034af757\" (UID: \"4e1fffa1-441f-4301-a2c4-f5c9034af757\") "
May 20 15:33:30.668042 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.667934    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4e1fffa1-441f-4301-a2c4-f5c9034af757" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 20 15:33:30.668098 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.668062    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4e1fffa1-441f-4301-a2c4-f5c9034af757" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 20 15:33:30.670542 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.670509    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4e1fffa1-441f-4301-a2c4-f5c9034af757" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:33:30.670677 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.670588    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4e1fffa1-441f-4301-a2c4-f5c9034af757" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:33:30.670728 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.670664    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-kube-api-access-sjz9n" (OuterVolumeSpecName: "kube-api-access-sjz9n") pod "4e1fffa1-441f-4301-a2c4-f5c9034af757" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757"). InnerVolumeSpecName "kube-api-access-sjz9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:33:30.670800 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.670724    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4e1fffa1-441f-4301-a2c4-f5c9034af757" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:33:30.670855 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.670836    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4e1fffa1-441f-4301-a2c4-f5c9034af757" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:33:30.679371 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.679339    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e1fffa1-441f-4301-a2c4-f5c9034af757-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4e1fffa1-441f-4301-a2c4-f5c9034af757" (UID: "4e1fffa1-441f-4301-a2c4-f5c9034af757"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:33:30.769289 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.769256    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-installation-pull-secrets\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:33:30.769289 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.769290    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-certificates\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:33:30.769485 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.769305    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-registry-tls\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:33:30.769485 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.769318    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e1fffa1-441f-4301-a2c4-f5c9034af757-trusted-ca\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:33:30.769485 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.769330    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-bound-sa-token\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:33:30.769485 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.769341    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e1fffa1-441f-4301-a2c4-f5c9034af757-ca-trust-extracted\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:33:30.769485 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.769353    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjz9n\" (UniqueName: \"kubernetes.io/projected/4e1fffa1-441f-4301-a2c4-f5c9034af757-kube-api-access-sjz9n\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:33:30.769485 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:30.769369    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e1fffa1-441f-4301-a2c4-f5c9034af757-image-registry-private-configuration\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:33:31.491177 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:31.491140    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-d5829" event={"ID":"055ee774-d1d8-4bd2-beee-c246461e5f65","Type":"ContainerStarted","Data":"fc9fc2ab82b133ff85f67571837cce5a93882549d3c82d8b6fc4c769302df4ef"}
May 20 15:33:31.491177 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:31.491173    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"
May 20 15:33:31.509595 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:31.509570    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"]
May 20 15:33:31.515412 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:31.515391    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68d6ddf9cc-hrh2c"]
May 20 15:33:32.935390 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:32.935355    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e1fffa1-441f-4301-a2c4-f5c9034af757" path="/var/lib/kubelet/pods/4e1fffa1-441f-4301-a2c4-f5c9034af757/volumes"
May 20 15:33:39.514163 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:39.514129    2566 generic.go:358] "Generic (PLEG): container finished" podID="12a104cf-a5bb-402e-8b9e-af91b6c3b1be" containerID="c96c0414da09e0a9b9faed0ef964fefc40ad930fe23633b6ab5c1003a0e0ccdb" exitCode=0
May 20 15:33:39.514564 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:39.514209    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc" event={"ID":"12a104cf-a5bb-402e-8b9e-af91b6c3b1be","Type":"ContainerDied","Data":"c96c0414da09e0a9b9faed0ef964fefc40ad930fe23633b6ab5c1003a0e0ccdb"}
May 20 15:33:39.514608 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:39.514566    2566 scope.go:117] "RemoveContainer" containerID="c96c0414da09e0a9b9faed0ef964fefc40ad930fe23633b6ab5c1003a0e0ccdb"
May 20 15:33:40.518342 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:40.518311    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-v4zqc" event={"ID":"12a104cf-a5bb-402e-8b9e-af91b6c3b1be","Type":"ContainerStarted","Data":"565f48a0a5a7a8ac9f19c23f3c157a41092630c98c42bc6f0b1ce23b5b00576b"}
May 20 15:33:57.474820 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:57.474776    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:33:57.491028 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:57.490996    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:33:57.584428 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:33:57.584394    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:15.527500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.527412    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 20 15:34:15.528246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.527929    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="prometheus" containerID="cri-o://a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3" gracePeriod=600
May 20 15:34:15.528246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.527960    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="thanos-sidecar" containerID="cri-o://edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86" gracePeriod=600
May 20 15:34:15.528246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.527990    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0" gracePeriod=600
May 20 15:34:15.528246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.527938    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy" containerID="cri-o://174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a" gracePeriod=600
May 20 15:34:15.528246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.528062    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy-web" containerID="cri-o://c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3" gracePeriod=600
May 20 15:34:15.528246 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.528128    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="config-reloader" containerID="cri-o://49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4" gracePeriod=600
May 20 15:34:15.769332 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.769310    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:15.842096 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842016    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-tls-assets\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842096 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842063    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-kubelet-serving-ca-bundle\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842096 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842093    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-rulefiles-0\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842118    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-tls\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842146    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config-out\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842206    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-metrics-client-ca\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842231    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-kube-rbac-proxy\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842268    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842303    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7cwv\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-kube-api-access-r7cwv\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842339    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-trusted-ca-bundle\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842365    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-web-config\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842395    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-thanos-prometheus-http-client-file\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842421    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-metrics-client-certs\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842455    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842483    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-db\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842510    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-serving-certs-ca-bundle\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842547    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.842686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842539    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 20 15:34:15.842686 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842574    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-grpc-tls\") pod \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\" (UID: \"94994f58-7484-4cf7-a2a6-70bab8b6f3e4\") "
May 20 15:34:15.843198 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.842835    2566 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.844205 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.844161    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 20 15:34:15.844356 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.844334    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 20 15:34:15.846305 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.846115    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 20 15:34:15.846305 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.846148    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 20 15:34:15.846305 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.846208    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:34:15.846305 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.846279    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config" (OuterVolumeSpecName: "config") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:34:15.846789 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.846739    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:34:15.846931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.846828    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:34:15.846931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.846825    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config-out" (OuterVolumeSpecName: "config-out") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:34:15.847167 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.847140    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:34:15.847666 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.847632    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-kube-api-access-r7cwv" (OuterVolumeSpecName: "kube-api-access-r7cwv") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "kube-api-access-r7cwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:34:15.847794 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.847737    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:34:15.847991 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.847973    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:34:15.848366 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.848344    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:34:15.848452 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.848412    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:34:15.848491 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.848455    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:34:15.858268 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.858246    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-web-config" (OuterVolumeSpecName: "web-config") pod "94994f58-7484-4cf7-a2a6-70bab8b6f3e4" (UID: "94994f58-7484-4cf7-a2a6-70bab8b6f3e4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:34:15.943946 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.943915    2566 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.943946 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.943939    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7cwv\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-kube-api-access-r7cwv\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.943946 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.943951    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.943961    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-web-config\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.943969    2566 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.943978    2566 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-metrics-client-certs\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.943988    2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.943997    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-db\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.944007    2566 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.944018    2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.944027    2566 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-grpc-tls\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.944035    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-tls-assets\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.944045    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.944054    2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.944062    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-config-out\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.944071    2566 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-configmap-metrics-client-ca\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:15.944147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:15.944079    2566 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94994f58-7484-4cf7-a2a6-70bab8b6f3e4-secret-kube-rbac-proxy\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:34:16.625774 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625721    2566 generic.go:358] "Generic (PLEG): container finished" podID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerID="2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0" exitCode=0
May 20 15:34:16.625774 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625761    2566 generic.go:358] "Generic (PLEG): container finished" podID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerID="174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a" exitCode=0
May 20 15:34:16.625774 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625768    2566 generic.go:358] "Generic (PLEG): container finished" podID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerID="c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3" exitCode=0
May 20 15:34:16.625774 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625775    2566 generic.go:358] "Generic (PLEG): container finished" podID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerID="edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86" exitCode=0
May 20 15:34:16.625774 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625780    2566 generic.go:358] "Generic (PLEG): container finished" podID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerID="49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4" exitCode=0
May 20 15:34:16.625774 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625785    2566 generic.go:358] "Generic (PLEG): container finished" podID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerID="a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3" exitCode=0
May 20 15:34:16.626415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625795    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerDied","Data":"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"}
May 20 15:34:16.626415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625841    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.626415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625856    2566 scope.go:117] "RemoveContainer" containerID="2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"
May 20 15:34:16.626415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625843    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerDied","Data":"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"}
May 20 15:34:16.626415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.625982    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerDied","Data":"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"}
May 20 15:34:16.626415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.626000    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerDied","Data":"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"}
May 20 15:34:16.626415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.626016    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerDied","Data":"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"}
May 20 15:34:16.626415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.626029    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerDied","Data":"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"}
May 20 15:34:16.626415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.626044    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"94994f58-7484-4cf7-a2a6-70bab8b6f3e4","Type":"ContainerDied","Data":"8fc603a1196c57e9fa1cf0868a9f1952a4a4b45ca26adb6ba71d2a9a1f57e68c"}
May 20 15:34:16.634967 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.634949    2566 scope.go:117] "RemoveContainer" containerID="174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"
May 20 15:34:16.642100 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.642084    2566 scope.go:117] "RemoveContainer" containerID="c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"
May 20 15:34:16.648726 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.648702    2566 scope.go:117] "RemoveContainer" containerID="edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"
May 20 15:34:16.650614 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.650591    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 20 15:34:16.656118 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.656097    2566 scope.go:117] "RemoveContainer" containerID="49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"
May 20 15:34:16.656956 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.656934    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 20 15:34:16.662785 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.662738    2566 scope.go:117] "RemoveContainer" containerID="a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"
May 20 15:34:16.669570 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.669555    2566 scope.go:117] "RemoveContainer" containerID="998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"
May 20 15:34:16.676084 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.676066    2566 scope.go:117] "RemoveContainer" containerID="2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"
May 20 15:34:16.676323 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:34:16.676303    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": container with ID starting with 2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0 not found: ID does not exist" containerID="2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"
May 20 15:34:16.676378 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.676332    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"} err="failed to get container status \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": rpc error: code = NotFound desc = could not find container \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": container with ID starting with 2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0 not found: ID does not exist"
May 20 15:34:16.676378 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.676368    2566 scope.go:117] "RemoveContainer" containerID="174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"
May 20 15:34:16.676603 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:34:16.676588    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": container with ID starting with 174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a not found: ID does not exist" containerID="174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"
May 20 15:34:16.676662 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.676606    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"} err="failed to get container status \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": rpc error: code = NotFound desc = could not find container \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": container with ID starting with 174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a not found: ID does not exist"
May 20 15:34:16.676662 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.676619    2566 scope.go:117] "RemoveContainer" containerID="c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"
May 20 15:34:16.676907 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:34:16.676891    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": container with ID starting with c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3 not found: ID does not exist" containerID="c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"
May 20 15:34:16.676950 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.676914    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"} err="failed to get container status \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": rpc error: code = NotFound desc = could not find container \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": container with ID starting with c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3 not found: ID does not exist"
May 20 15:34:16.676950 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.676936    2566 scope.go:117] "RemoveContainer" containerID="edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"
May 20 15:34:16.677135 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:34:16.677120    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": container with ID starting with edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86 not found: ID does not exist" containerID="edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"
May 20 15:34:16.677176 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.677139    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"} err="failed to get container status \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": rpc error: code = NotFound desc = could not find container \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": container with ID starting with edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86 not found: ID does not exist"
May 20 15:34:16.677176 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.677153    2566 scope.go:117] "RemoveContainer" containerID="49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"
May 20 15:34:16.677349 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:34:16.677335    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": container with ID starting with 49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4 not found: ID does not exist" containerID="49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"
May 20 15:34:16.677386 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.677354    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"} err="failed to get container status \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": rpc error: code = NotFound desc = could not find container \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": container with ID starting with 49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4 not found: ID does not exist"
May 20 15:34:16.677386 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.677368    2566 scope.go:117] "RemoveContainer" containerID="a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"
May 20 15:34:16.677563 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:34:16.677541    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": container with ID starting with a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3 not found: ID does not exist" containerID="a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"
May 20 15:34:16.677608 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.677568    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"} err="failed to get container status \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": rpc error: code = NotFound desc = could not find container \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": container with ID starting with a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3 not found: ID does not exist"
May 20 15:34:16.677608 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.677581    2566 scope.go:117] "RemoveContainer" containerID="998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"
May 20 15:34:16.677824 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:34:16.677800    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": container with ID starting with 998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51 not found: ID does not exist" containerID="998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"
May 20 15:34:16.677923 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.677826    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"} err="failed to get container status \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": rpc error: code = NotFound desc = could not find container \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": container with ID starting with 998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51 not found: ID does not exist"
May 20 15:34:16.677923 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.677842    2566 scope.go:117] "RemoveContainer" containerID="2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"
May 20 15:34:16.678103 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678088    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"} err="failed to get container status \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": rpc error: code = NotFound desc = could not find container \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": container with ID starting with 2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0 not found: ID does not exist"
May 20 15:34:16.678103 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678103    2566 scope.go:117] "RemoveContainer" containerID="174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"
May 20 15:34:16.678317 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678300    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"} err="failed to get container status \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": rpc error: code = NotFound desc = could not find container \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": container with ID starting with 174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a not found: ID does not exist"
May 20 15:34:16.678367 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678318    2566 scope.go:117] "RemoveContainer" containerID="c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"
May 20 15:34:16.678533 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678517    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"} err="failed to get container status \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": rpc error: code = NotFound desc = could not find container \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": container with ID starting with c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3 not found: ID does not exist"
May 20 15:34:16.678533 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678532    2566 scope.go:117] "RemoveContainer" containerID="edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"
May 20 15:34:16.678727 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678711    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"} err="failed to get container status \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": rpc error: code = NotFound desc = could not find container \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": container with ID starting with edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86 not found: ID does not exist"
May 20 15:34:16.678805 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678727    2566 scope.go:117] "RemoveContainer" containerID="49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"
May 20 15:34:16.678940 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678925    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"} err="failed to get container status \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": rpc error: code = NotFound desc = could not find container \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": container with ID starting with 49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4 not found: ID does not exist"
May 20 15:34:16.678990 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.678940    2566 scope.go:117] "RemoveContainer" containerID="a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"
May 20 15:34:16.679121 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.679101    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"} err="failed to get container status \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": rpc error: code = NotFound desc = could not find container \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": container with ID starting with a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3 not found: ID does not exist"
May 20 15:34:16.679166 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.679122    2566 scope.go:117] "RemoveContainer" containerID="998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"
May 20 15:34:16.679328 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.679304    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"} err="failed to get container status \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": rpc error: code = NotFound desc = could not find container \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": container with ID starting with 998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51 not found: ID does not exist"
May 20 15:34:16.679328 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.679328    2566 scope.go:117] "RemoveContainer" containerID="2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"
May 20 15:34:16.679683 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.679640    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"} err="failed to get container status \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": rpc error: code = NotFound desc = could not find container \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": container with ID starting with 2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0 not found: ID does not exist"
May 20 15:34:16.679683 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.679669    2566 scope.go:117] "RemoveContainer" containerID="174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"
May 20 15:34:16.680097 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.679932    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"} err="failed to get container status \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": rpc error: code = NotFound desc = could not find container \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": container with ID starting with 174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a not found: ID does not exist"
May 20 15:34:16.680097 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.679956    2566 scope.go:117] "RemoveContainer" containerID="c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"
May 20 15:34:16.680404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.680369    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"} err="failed to get container status \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": rpc error: code = NotFound desc = could not find container \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": container with ID starting with c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3 not found: ID does not exist"
May 20 15:34:16.680518 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.680408    2566 scope.go:117] "RemoveContainer" containerID="edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"
May 20 15:34:16.680687 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.680657    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"} err="failed to get container status \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": rpc error: code = NotFound desc = could not find container \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": container with ID starting with edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86 not found: ID does not exist"
May 20 15:34:16.680775 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.680689    2566 scope.go:117] "RemoveContainer" containerID="49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"
May 20 15:34:16.680936 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.680917    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"} err="failed to get container status \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": rpc error: code = NotFound desc = could not find container \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": container with ID starting with 49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4 not found: ID does not exist"
May 20 15:34:16.680995 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.680938    2566 scope.go:117] "RemoveContainer" containerID="a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"
May 20 15:34:16.681114 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681093    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"} err="failed to get container status \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": rpc error: code = NotFound desc = could not find container \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": container with ID starting with a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3 not found: ID does not exist"
May 20 15:34:16.681114 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681113    2566 scope.go:117] "RemoveContainer" containerID="998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"
May 20 15:34:16.681326 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681311    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 20 15:34:16.681408 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681310    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"} err="failed to get container status \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": rpc error: code = NotFound desc = could not find container \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": container with ID starting with 998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51 not found: ID does not exist"
May 20 15:34:16.681445 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681413    2566 scope.go:117] "RemoveContainer" containerID="2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"
May 20 15:34:16.681633 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681616    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="config-reloader"
May 20 15:34:16.681693 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681636    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="config-reloader"
May 20 15:34:16.681693 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681652    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="prometheus"
May 20 15:34:16.681693 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681662    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="prometheus"
May 20 15:34:16.681693 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681672    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy-web"
May 20 15:34:16.681693 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681664    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"} err="failed to get container status \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": rpc error: code = NotFound desc = could not find container \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": container with ID starting with 2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0 not found: ID does not exist"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681694    2566 scope.go:117] "RemoveContainer" containerID="174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681678    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy-web"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681776    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681788    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681812    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="init-config-reloader"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681821    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="init-config-reloader"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681830    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy-thanos"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681838    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy-thanos"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681849    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e1fffa1-441f-4301-a2c4-f5c9034af757" containerName="registry"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681857    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e1fffa1-441f-4301-a2c4-f5c9034af757" containerName="registry"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681870    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="thanos-sidecar"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681877    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="thanos-sidecar"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681959    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="thanos-sidecar"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681973    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="4e1fffa1-441f-4301-a2c4-f5c9034af757" containerName="registry"
May 20 15:34:16.681971 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681980    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy"
May 20 15:34:16.682446 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681988    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="config-reloader"
May 20 15:34:16.682446 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681995    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="prometheus"
May 20 15:34:16.682446 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682003    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy-thanos"
May 20 15:34:16.682446 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682009    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" containerName="kube-rbac-proxy-web"
May 20 15:34:16.682446 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.681973    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"} err="failed to get container status \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": rpc error: code = NotFound desc = could not find container \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": container with ID starting with 174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a not found: ID does not exist"
May 20 15:34:16.682446 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682050    2566 scope.go:117] "RemoveContainer" containerID="c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"
May 20 15:34:16.682446 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682282    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"} err="failed to get container status \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": rpc error: code = NotFound desc = could not find container \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": container with ID starting with c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3 not found: ID does not exist"
May 20 15:34:16.682446 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682298    2566 scope.go:117] "RemoveContainer" containerID="edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"
May 20 15:34:16.682813 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682481    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"} err="failed to get container status \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": rpc error: code = NotFound desc = could not find container \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": container with ID starting with edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86 not found: ID does not exist"
May 20 15:34:16.682813 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682506    2566 scope.go:117] "RemoveContainer" containerID="49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"
May 20 15:34:16.682813 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682693    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"} err="failed to get container status \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": rpc error: code = NotFound desc = could not find container \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": container with ID starting with 49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4 not found: ID does not exist"
May 20 15:34:16.682813 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682712    2566 scope.go:117] "RemoveContainer" containerID="a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"
May 20 15:34:16.682978 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682948    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"} err="failed to get container status \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": rpc error: code = NotFound desc = could not find container \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": container with ID starting with a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3 not found: ID does not exist"
May 20 15:34:16.682978 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.682966    2566 scope.go:117] "RemoveContainer" containerID="998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"
May 20 15:34:16.683182 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683167    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"} err="failed to get container status \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": rpc error: code = NotFound desc = could not find container \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": container with ID starting with 998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51 not found: ID does not exist"
May 20 15:34:16.683227 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683184    2566 scope.go:117] "RemoveContainer" containerID="2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"
May 20 15:34:16.683368 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683352    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"} err="failed to get container status \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": rpc error: code = NotFound desc = could not find container \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": container with ID starting with 2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0 not found: ID does not exist"
May 20 15:34:16.683409 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683368    2566 scope.go:117] "RemoveContainer" containerID="174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"
May 20 15:34:16.683545 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683528    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"} err="failed to get container status \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": rpc error: code = NotFound desc = could not find container \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": container with ID starting with 174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a not found: ID does not exist"
May 20 15:34:16.683588 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683545    2566 scope.go:117] "RemoveContainer" containerID="c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"
May 20 15:34:16.683762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683698    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"} err="failed to get container status \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": rpc error: code = NotFound desc = could not find container \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": container with ID starting with c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3 not found: ID does not exist"
May 20 15:34:16.683762 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683714    2566 scope.go:117] "RemoveContainer" containerID="edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"
May 20 15:34:16.683963 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683944    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"} err="failed to get container status \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": rpc error: code = NotFound desc = could not find container \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": container with ID starting with edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86 not found: ID does not exist"
May 20 15:34:16.684035 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.683966    2566 scope.go:117] "RemoveContainer" containerID="49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"
May 20 15:34:16.684192 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.684174    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"} err="failed to get container status \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": rpc error: code = NotFound desc = could not find container \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": container with ID starting with 49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4 not found: ID does not exist"
May 20 15:34:16.684192 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.684201    2566 scope.go:117] "RemoveContainer" containerID="a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"
May 20 15:34:16.684424 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.684407    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"} err="failed to get container status \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": rpc error: code = NotFound desc = could not find container \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": container with ID starting with a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3 not found: ID does not exist"
May 20 15:34:16.684470 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.684424    2566 scope.go:117] "RemoveContainer" containerID="998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"
May 20 15:34:16.684625 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.684607    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"} err="failed to get container status \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": rpc error: code = NotFound desc = could not find container \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": container with ID starting with 998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51 not found: ID does not exist"
May 20 15:34:16.684693 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.684628    2566 scope.go:117] "RemoveContainer" containerID="2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"
May 20 15:34:16.684870 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.684854    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0"} err="failed to get container status \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": rpc error: code = NotFound desc = could not find container \"2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0\": container with ID starting with 2eb97a6ebf593168b34989e8b43f63e17a806751431ff930baba4a17238deba0 not found: ID does not exist"
May 20 15:34:16.684870 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.684869    2566 scope.go:117] "RemoveContainer" containerID="174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"
May 20 15:34:16.685066 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685048    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a"} err="failed to get container status \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": rpc error: code = NotFound desc = could not find container \"174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a\": container with ID starting with 174448ae2cf37762404274b0aad9a5d25181faf74b7c722cc2e2cd7029b53d1a not found: ID does not exist"
May 20 15:34:16.685066 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685062    2566 scope.go:117] "RemoveContainer" containerID="c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"
May 20 15:34:16.685262 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685242    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3"} err="failed to get container status \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": rpc error: code = NotFound desc = could not find container \"c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3\": container with ID starting with c3fda1f4520a0809bfa9f1bc9b87c8c273cee30e1910eb6e31bbc1f0bee804f3 not found: ID does not exist"
May 20 15:34:16.685311 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685263    2566 scope.go:117] "RemoveContainer" containerID="edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"
May 20 15:34:16.685449 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685429    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86"} err="failed to get container status \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": rpc error: code = NotFound desc = could not find container \"edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86\": container with ID starting with edb7d94a433929a864b72f9bbdc14802820a1270aaaadecae4c07c39a970cc86 not found: ID does not exist"
May 20 15:34:16.685512 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685452    2566 scope.go:117] "RemoveContainer" containerID="49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"
May 20 15:34:16.685632 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685614    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4"} err="failed to get container status \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": rpc error: code = NotFound desc = could not find container \"49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4\": container with ID starting with 49460b43bf4b1358378d22c97b1dc92d05004cbc65b265bf5956ec890dd911c4 not found: ID does not exist"
May 20 15:34:16.685678 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685635    2566 scope.go:117] "RemoveContainer" containerID="a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"
May 20 15:34:16.685715 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685701    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.686084 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685922    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3"} err="failed to get container status \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": rpc error: code = NotFound desc = could not find container \"a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3\": container with ID starting with a289b9ccfbd20fe2b696107e0759a18bb6d3bf2985381b2fb41c50441fb88fb3 not found: ID does not exist"
May 20 15:34:16.686084 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.685950    2566 scope.go:117] "RemoveContainer" containerID="998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"
May 20 15:34:16.686281 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.686262    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51"} err="failed to get container status \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": rpc error: code = NotFound desc = could not find container \"998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51\": container with ID starting with 998e88ac4caee6bde825599852441ba49e07908140cfb2dd2c2a2dd6cf278c51 not found: ID does not exist"
May 20 15:34:16.688349 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.688322    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2j4elu8qe5nrl\""
May 20 15:34:16.688349 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.688341    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 20 15:34:16.688349 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.688356    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9g2kd\""
May 20 15:34:16.688561 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.688355    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 20 15:34:16.688561 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.688465    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 20 15:34:16.688704 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.688689    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 20 15:34:16.688812 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.688796    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 20 15:34:16.688891 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.688803    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 20 15:34:16.688891 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.688887    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 20 15:34:16.689101 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.689087    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 20 15:34:16.689434 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.689417    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 20 15:34:16.689510 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.689420    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 20 15:34:16.689510 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.689420    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 20 15:34:16.693802 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.693774    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 20 15:34:16.695422 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.695405    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 20 15:34:16.700032 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.700008    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 20 15:34:16.750529 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750492    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750529 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750524    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-config\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750550    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750566    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750626    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750686    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750706    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750723    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-web-config\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750960 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750741    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750960 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750793    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750960 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750811    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phltm\" (UniqueName: \"kubernetes.io/projected/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-kube-api-access-phltm\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750960 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750829    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750960 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750844    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750960 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750860    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750960 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750895    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750960 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750914    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-config-out\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.750960 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.750949    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.851986 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.851900    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.851986 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.851949    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.851986 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.851983    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852003    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-config-out\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852127    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852270 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852204    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852270 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852234    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-config\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852352 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852275    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852352 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852303    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852352 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852335    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852510 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852364    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852510 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852421    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852510 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852452    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852510 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852478    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-web-config\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852703 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852540    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852703 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852575    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852703 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852635    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-phltm\" (UniqueName: \"kubernetes.io/projected/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-kube-api-access-phltm\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.852703 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852669    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.853094 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.852794    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.853633 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.853580    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.856450 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.855166    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-config-out\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.856450 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.855340    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.856450 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.855545    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.856450 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.855585    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.856450 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.855593    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.856787 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.856494    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.857481 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.857164    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.857481 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.857429    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.857740 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.857714    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.857827 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.857771    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-config\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.858090 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.858064    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.858248 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.858230    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.858304 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.858226    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.858492 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.858469    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.859482 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.859465    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-web-config\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.866565 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.866541    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phltm\" (UniqueName: \"kubernetes.io/projected/b3ed908a-4eb4-4a80-8ab1-708e23e4ea00-kube-api-access-phltm\") pod \"prometheus-k8s-0\" (UID: \"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00\") " pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:16.933504 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.933455    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94994f58-7484-4cf7-a2a6-70bab8b6f3e4" path="/var/lib/kubelet/pods/94994f58-7484-4cf7-a2a6-70bab8b6f3e4/volumes"
May 20 15:34:16.995303 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:16.995257    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:34:17.131335 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:17.131312    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 20 15:34:17.133243 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:34:17.133216    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ed908a_4eb4_4a80_8ab1_708e23e4ea00.slice/crio-4914b095c6659aaeec475f9d6e31b0df5699bcdc6a151c59fe9652dba8418b7f WatchSource:0}: Error finding container 4914b095c6659aaeec475f9d6e31b0df5699bcdc6a151c59fe9652dba8418b7f: Status 404 returned error can't find the container with id 4914b095c6659aaeec475f9d6e31b0df5699bcdc6a151c59fe9652dba8418b7f
May 20 15:34:17.630705 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:17.630670    2566 generic.go:358] "Generic (PLEG): container finished" podID="b3ed908a-4eb4-4a80-8ab1-708e23e4ea00" containerID="87c87004bfa21177c81471d058bcd2132911e9c1d96d5a3d3f28dffbf73dc1b9" exitCode=0
May 20 15:34:17.631171 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:17.630726    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00","Type":"ContainerDied","Data":"87c87004bfa21177c81471d058bcd2132911e9c1d96d5a3d3f28dffbf73dc1b9"}
May 20 15:34:17.631171 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:17.630765    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00","Type":"ContainerStarted","Data":"4914b095c6659aaeec475f9d6e31b0df5699bcdc6a151c59fe9652dba8418b7f"}
May 20 15:34:18.636823 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:18.636791    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00","Type":"ContainerStarted","Data":"51d47db62e5d127730f0233544b5a2a40e686cdbfa552e47e627a24d6866b5a7"}
May 20 15:34:18.636823 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:18.636825    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00","Type":"ContainerStarted","Data":"30faa1ecc850e990dc4ca5d6cb6c10a8b216cbac0c43ab34b2e8ed39794ac80b"}
May 20 15:34:18.637227 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:18.636843    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00","Type":"ContainerStarted","Data":"c43cff1a5b60d671ea4e64b3791a1adddf99c888057989a475a60921db6e5363"}
May 20 15:34:18.637227 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:18.636853    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00","Type":"ContainerStarted","Data":"1a677bf4464ebd30efc98dd645495dac8e64a351c4bd1810d3d6a1dca9f97092"}
May 20 15:34:18.637227 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:18.636861    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00","Type":"ContainerStarted","Data":"6d863de2560ca66a074de12a7cab9339ae6ab641bc7369fb8b13c68167a10de1"}
May 20 15:34:18.637227 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:18.636872    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b3ed908a-4eb4-4a80-8ab1-708e23e4ea00","Type":"ContainerStarted","Data":"8ea08b913bb79ae99ac3de9c686d7efa5345ee84a5f40415469acd8eedff99f2"}
May 20 15:34:18.665631 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:18.665551    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.665531293 podStartE2EDuration="2.665531293s" podCreationTimestamp="2026-05-20 15:34:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:34:18.663529928 +0000 UTC m=+168.347399268" watchObservedRunningTime="2026-05-20 15:34:18.665531293 +0000 UTC m=+168.349400631"
May 20 15:34:21.996118 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:34:21.996076    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:35:16.995928 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:35:16.995886    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:35:17.012330 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:35:17.012304    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:35:17.828117 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:35:17.828089    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 20 15:36:30.835707 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:36:30.835676    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:36:30.837766 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:36:30.837719    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:36:30.842117 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:36:30.842095    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:36:30.844475 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:36:30.844456    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:36:30.844925 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:36:30.844907    2566 kubelet.go:1628] "Image garbage collection succeeded"
May 20 15:39:30.864830 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:39:30.864797    2566 scope.go:117] "RemoveContainer" containerID="a88d351b518cabc8130a9e00ac82f9ae7c53a6b5b22e4688f4b5a6ae908cb299"
May 20 15:40:58.079052 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.079013    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-722cm"]
May 20 15:40:58.082221 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.082201    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:40:58.084688 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.084662    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
May 20 15:40:58.085371 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.085354    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
May 20 15:40:58.085432 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.085354    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-hgm7v\""
May 20 15:40:58.092519 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.092500    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-722cm"]
May 20 15:40:58.182511 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.182471    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7hmb\" (UniqueName: \"kubernetes.io/projected/3ae989fe-aa9e-447a-b5f5-6107736013e6-kube-api-access-b7hmb\") pod \"cert-manager-webhook-587ccfb98-722cm\" (UID: \"3ae989fe-aa9e-447a-b5f5-6107736013e6\") " pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:40:58.182683 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.182524    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ae989fe-aa9e-447a-b5f5-6107736013e6-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-722cm\" (UID: \"3ae989fe-aa9e-447a-b5f5-6107736013e6\") " pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:40:58.283588 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.283555    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b7hmb\" (UniqueName: \"kubernetes.io/projected/3ae989fe-aa9e-447a-b5f5-6107736013e6-kube-api-access-b7hmb\") pod \"cert-manager-webhook-587ccfb98-722cm\" (UID: \"3ae989fe-aa9e-447a-b5f5-6107736013e6\") " pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:40:58.283742 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.283596    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ae989fe-aa9e-447a-b5f5-6107736013e6-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-722cm\" (UID: \"3ae989fe-aa9e-447a-b5f5-6107736013e6\") " pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:40:58.292430 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.292401    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7hmb\" (UniqueName: \"kubernetes.io/projected/3ae989fe-aa9e-447a-b5f5-6107736013e6-kube-api-access-b7hmb\") pod \"cert-manager-webhook-587ccfb98-722cm\" (UID: \"3ae989fe-aa9e-447a-b5f5-6107736013e6\") " pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:40:58.292649 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.292628    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ae989fe-aa9e-447a-b5f5-6107736013e6-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-722cm\" (UID: \"3ae989fe-aa9e-447a-b5f5-6107736013e6\") " pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:40:58.398731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.398647    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:40:58.527207 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.527184    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-722cm"]
May 20 15:40:58.529948 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:40:58.529921    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ae989fe_aa9e_447a_b5f5_6107736013e6.slice/crio-7e3444e4cbcd2ee21cf0eccd617398a0e991f5991c14d78165dadc3c575cf0df WatchSource:0}: Error finding container 7e3444e4cbcd2ee21cf0eccd617398a0e991f5991c14d78165dadc3c575cf0df: Status 404 returned error can't find the container with id 7e3444e4cbcd2ee21cf0eccd617398a0e991f5991c14d78165dadc3c575cf0df
May 20 15:40:58.531676 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.531657    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 20 15:40:58.799581 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:40:58.799548    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-722cm" event={"ID":"3ae989fe-aa9e-447a-b5f5-6107736013e6","Type":"ContainerStarted","Data":"7e3444e4cbcd2ee21cf0eccd617398a0e991f5991c14d78165dadc3c575cf0df"}
May 20 15:41:01.810893 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:01.810860    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-722cm" event={"ID":"3ae989fe-aa9e-447a-b5f5-6107736013e6","Type":"ContainerStarted","Data":"a994ae71ff3582e08e5ffab8174e25b7fd0b72d7563528e763cebea4e9a1f164"}
May 20 15:41:01.811308 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:01.810979    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:41:01.828962 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:01.828906    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-722cm" podStartSLOduration=0.772452002 podStartE2EDuration="3.828887883s" podCreationTimestamp="2026-05-20 15:40:58 +0000 UTC" firstStartedPulling="2026-05-20 15:40:58.531837516 +0000 UTC m=+568.215706833" lastFinishedPulling="2026-05-20 15:41:01.588273386 +0000 UTC m=+571.272142714" observedRunningTime="2026-05-20 15:41:01.828507771 +0000 UTC m=+571.512377122" watchObservedRunningTime="2026-05-20 15:41:01.828887883 +0000 UTC m=+571.512757224"
May 20 15:41:07.815940 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:07.815905    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-722cm"
May 20 15:41:12.467634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.467599    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"]
May 20 15:41:12.470740 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.470724    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"
May 20 15:41:12.473233 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.473209    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
May 20 15:41:12.474208 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.474187    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
May 20 15:41:12.474303 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.474225    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-x2hc9\""
May 20 15:41:12.478731 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.478712    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"]
May 20 15:41:12.508240 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.508211    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b41e364-37af-4c7e-ad47-2ea95de7bd3f-tmp\") pod \"openshift-lws-operator-bfc7f696d-9646h\" (UID: \"3b41e364-37af-4c7e-ad47-2ea95de7bd3f\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"
May 20 15:41:12.508353 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.508271    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg8mj\" (UniqueName: \"kubernetes.io/projected/3b41e364-37af-4c7e-ad47-2ea95de7bd3f-kube-api-access-bg8mj\") pod \"openshift-lws-operator-bfc7f696d-9646h\" (UID: \"3b41e364-37af-4c7e-ad47-2ea95de7bd3f\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"
May 20 15:41:12.609593 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.609558    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b41e364-37af-4c7e-ad47-2ea95de7bd3f-tmp\") pod \"openshift-lws-operator-bfc7f696d-9646h\" (UID: \"3b41e364-37af-4c7e-ad47-2ea95de7bd3f\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"
May 20 15:41:12.609742 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.609620    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bg8mj\" (UniqueName: \"kubernetes.io/projected/3b41e364-37af-4c7e-ad47-2ea95de7bd3f-kube-api-access-bg8mj\") pod \"openshift-lws-operator-bfc7f696d-9646h\" (UID: \"3b41e364-37af-4c7e-ad47-2ea95de7bd3f\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"
May 20 15:41:12.609966 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.609945    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b41e364-37af-4c7e-ad47-2ea95de7bd3f-tmp\") pod \"openshift-lws-operator-bfc7f696d-9646h\" (UID: \"3b41e364-37af-4c7e-ad47-2ea95de7bd3f\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"
May 20 15:41:12.618626 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.618608    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg8mj\" (UniqueName: \"kubernetes.io/projected/3b41e364-37af-4c7e-ad47-2ea95de7bd3f-kube-api-access-bg8mj\") pod \"openshift-lws-operator-bfc7f696d-9646h\" (UID: \"3b41e364-37af-4c7e-ad47-2ea95de7bd3f\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"
May 20 15:41:12.780429 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.780396    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"
May 20 15:41:12.911261 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:12.911233    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h"]
May 20 15:41:12.914007 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:41:12.913980    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b41e364_37af_4c7e_ad47_2ea95de7bd3f.slice/crio-b4364c9b05f97e2fe56755790561e22a5e51e959ecfe6e78947dd3081f16c71e WatchSource:0}: Error finding container b4364c9b05f97e2fe56755790561e22a5e51e959ecfe6e78947dd3081f16c71e: Status 404 returned error can't find the container with id b4364c9b05f97e2fe56755790561e22a5e51e959ecfe6e78947dd3081f16c71e
May 20 15:41:13.851417 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:13.851366    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h" event={"ID":"3b41e364-37af-4c7e-ad47-2ea95de7bd3f","Type":"ContainerStarted","Data":"b4364c9b05f97e2fe56755790561e22a5e51e959ecfe6e78947dd3081f16c71e"}
May 20 15:41:15.858629 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:15.858546    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h" event={"ID":"3b41e364-37af-4c7e-ad47-2ea95de7bd3f","Type":"ContainerStarted","Data":"bb97d1e0139c1c40b62a6cf03246364e2971ba88f0ddd8630185e44e37764f5c"}
May 20 15:41:15.877105 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:15.877055    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9646h" podStartSLOduration=1.274304841 podStartE2EDuration="3.877038103s" podCreationTimestamp="2026-05-20 15:41:12 +0000 UTC" firstStartedPulling="2026-05-20 15:41:12.915550535 +0000 UTC m=+582.599419853" lastFinishedPulling="2026-05-20 15:41:15.518283799 +0000 UTC m=+585.202153115" observedRunningTime="2026-05-20 15:41:15.875661469 +0000 UTC m=+585.559530809" watchObservedRunningTime="2026-05-20 15:41:15.877038103 +0000 UTC m=+585.560907441"
May 20 15:41:30.860383 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:30.860346    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:41:30.860950 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:30.860439    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:41:30.869327 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:30.869307    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:41:30.869440 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:30.869309    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:41:40.200549 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.200513    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"]
May 20 15:41:40.205140 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.205121    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.209947 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.209923    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xd2hj\""
May 20 15:41:40.210088 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.209929    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
May 20 15:41:40.210088 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.209929    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
May 20 15:41:40.210088 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.209929    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
May 20 15:41:40.217805 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.217783    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"]
May 20 15:41:40.353758 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.353718    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z4d2\" (UniqueName: \"kubernetes.io/projected/29e5343b-74c2-4df6-ad9c-a0509521d761-kube-api-access-7z4d2\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.353929 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.353807    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29e5343b-74c2-4df6-ad9c-a0509521d761-cert\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.353929 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.353862    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/29e5343b-74c2-4df6-ad9c-a0509521d761-metrics-cert\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.354002 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.353947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29e5343b-74c2-4df6-ad9c-a0509521d761-manager-config\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.455137 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.455054    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29e5343b-74c2-4df6-ad9c-a0509521d761-manager-config\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.455137 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.455091    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7z4d2\" (UniqueName: \"kubernetes.io/projected/29e5343b-74c2-4df6-ad9c-a0509521d761-kube-api-access-7z4d2\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.455137 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.455118    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29e5343b-74c2-4df6-ad9c-a0509521d761-cert\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.455344 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.455184    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/29e5343b-74c2-4df6-ad9c-a0509521d761-metrics-cert\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.455825 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.455800    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29e5343b-74c2-4df6-ad9c-a0509521d761-manager-config\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.457853 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.457834    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29e5343b-74c2-4df6-ad9c-a0509521d761-cert\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.457996 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.457975    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/29e5343b-74c2-4df6-ad9c-a0509521d761-metrics-cert\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.466571 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.466550    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z4d2\" (UniqueName: \"kubernetes.io/projected/29e5343b-74c2-4df6-ad9c-a0509521d761-kube-api-access-7z4d2\") pod \"lws-controller-manager-cf55c5997-mrlb7\" (UID: \"29e5343b-74c2-4df6-ad9c-a0509521d761\") " pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.514459 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.514428    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:40.677869 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.677834    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"]
May 20 15:41:40.679024 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:41:40.678997    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e5343b_74c2_4df6_ad9c_a0509521d761.slice/crio-3c43e9610384afd166b53059506e761dc8fc17b91ad935506d3bf856a6d79b0d WatchSource:0}: Error finding container 3c43e9610384afd166b53059506e761dc8fc17b91ad935506d3bf856a6d79b0d: Status 404 returned error can't find the container with id 3c43e9610384afd166b53059506e761dc8fc17b91ad935506d3bf856a6d79b0d
May 20 15:41:40.931345 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:40.931308    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7" event={"ID":"29e5343b-74c2-4df6-ad9c-a0509521d761","Type":"ContainerStarted","Data":"3c43e9610384afd166b53059506e761dc8fc17b91ad935506d3bf856a6d79b0d"}
May 20 15:41:43.940853 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:43.940802    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7" event={"ID":"29e5343b-74c2-4df6-ad9c-a0509521d761","Type":"ContainerStarted","Data":"cb86dff82707836d78fc29934a093e4425c4ab31f03ee16a2ab272b995273f0e"}
May 20 15:41:43.941283 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:43.940921    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:41:43.958145 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:43.958102    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7" podStartSLOduration=1.7211277759999999 podStartE2EDuration="3.958090906s" podCreationTimestamp="2026-05-20 15:41:40 +0000 UTC" firstStartedPulling="2026-05-20 15:41:40.68073204 +0000 UTC m=+610.364601358" lastFinishedPulling="2026-05-20 15:41:42.917695164 +0000 UTC m=+612.601564488" observedRunningTime="2026-05-20 15:41:43.957376984 +0000 UTC m=+613.641246323" watchObservedRunningTime="2026-05-20 15:41:43.958090906 +0000 UTC m=+613.641960245"
May 20 15:41:54.945974 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:41:54.945943    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-cf55c5997-mrlb7"
May 20 15:42:33.418564 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.418526    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"]
May 20 15:42:33.422119 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.422099    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"
May 20 15:42:33.428654 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.428284    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
May 20 15:42:33.428781 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.428694    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 20 15:42:33.429241 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.429224    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 20 15:42:33.429328 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.429231    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sbcdd\""
May 20 15:42:33.435443 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.435420    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"]
May 20 15:42:33.597114 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.597077    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62ljf\" (UniqueName: \"kubernetes.io/projected/34e00de5-0e24-4f1c-94d8-f66277ec92ea-kube-api-access-62ljf\") pod \"dns-operator-controller-manager-844548ff4c-tt77j\" (UID: \"34e00de5-0e24-4f1c-94d8-f66277ec92ea\") " pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"
May 20 15:42:33.698228 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.698127    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-62ljf\" (UniqueName: \"kubernetes.io/projected/34e00de5-0e24-4f1c-94d8-f66277ec92ea-kube-api-access-62ljf\") pod \"dns-operator-controller-manager-844548ff4c-tt77j\" (UID: \"34e00de5-0e24-4f1c-94d8-f66277ec92ea\") " pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"
May 20 15:42:33.706765 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.706717    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62ljf\" (UniqueName: \"kubernetes.io/projected/34e00de5-0e24-4f1c-94d8-f66277ec92ea-kube-api-access-62ljf\") pod \"dns-operator-controller-manager-844548ff4c-tt77j\" (UID: \"34e00de5-0e24-4f1c-94d8-f66277ec92ea\") " pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"
May 20 15:42:33.731677 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.731647    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"
May 20 15:42:33.865438 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:33.865413    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"]
May 20 15:42:33.868173 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:42:33.868143    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e00de5_0e24_4f1c_94d8_f66277ec92ea.slice/crio-7141693cb17294c74ae4fe9af14323f72f55ff416c29cf42e83dcd06caa2e317 WatchSource:0}: Error finding container 7141693cb17294c74ae4fe9af14323f72f55ff416c29cf42e83dcd06caa2e317: Status 404 returned error can't find the container with id 7141693cb17294c74ae4fe9af14323f72f55ff416c29cf42e83dcd06caa2e317
May 20 15:42:34.085714 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:34.085675    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j" event={"ID":"34e00de5-0e24-4f1c-94d8-f66277ec92ea","Type":"ContainerStarted","Data":"7141693cb17294c74ae4fe9af14323f72f55ff416c29cf42e83dcd06caa2e317"}
May 20 15:42:37.097689 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:37.097655    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j" event={"ID":"34e00de5-0e24-4f1c-94d8-f66277ec92ea","Type":"ContainerStarted","Data":"836bafeea475319b8f7d2f5414638c8874cd4a9db879375a30adb7dc98d65d95"}
May 20 15:42:37.098092 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:37.097797    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"
May 20 15:42:37.131453 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:37.131399    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j" podStartSLOduration=1.279392408 podStartE2EDuration="4.131384434s" podCreationTimestamp="2026-05-20 15:42:33 +0000 UTC" firstStartedPulling="2026-05-20 15:42:33.870202451 +0000 UTC m=+663.554071767" lastFinishedPulling="2026-05-20 15:42:36.722194474 +0000 UTC m=+666.406063793" observedRunningTime="2026-05-20 15:42:37.125194426 +0000 UTC m=+666.809063765" watchObservedRunningTime="2026-05-20 15:42:37.131384434 +0000 UTC m=+666.815253772"
May 20 15:42:41.503057 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.503017    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"]
May 20 15:42:41.506128 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.506113    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:42:41.510371 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.510355    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mxrtm\""
May 20 15:42:41.529285 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.529253    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"]
May 20 15:42:41.559525 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.559489    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92323125-b9ec-4231-beb8-180abd22ab0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57dd85bb4-nclhb\" (UID: \"92323125-b9ec-4231-beb8-180abd22ab0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:42:41.559664 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.559564    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck7ph\" (UniqueName: \"kubernetes.io/projected/92323125-b9ec-4231-beb8-180abd22ab0f-kube-api-access-ck7ph\") pod \"kuadrant-operator-controller-manager-57dd85bb4-nclhb\" (UID: \"92323125-b9ec-4231-beb8-180abd22ab0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:42:41.660743 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.660694    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92323125-b9ec-4231-beb8-180abd22ab0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57dd85bb4-nclhb\" (UID: \"92323125-b9ec-4231-beb8-180abd22ab0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:42:41.660925 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.660812    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ck7ph\" (UniqueName: \"kubernetes.io/projected/92323125-b9ec-4231-beb8-180abd22ab0f-kube-api-access-ck7ph\") pod \"kuadrant-operator-controller-manager-57dd85bb4-nclhb\" (UID: \"92323125-b9ec-4231-beb8-180abd22ab0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:42:41.661136 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.661112    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92323125-b9ec-4231-beb8-180abd22ab0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57dd85bb4-nclhb\" (UID: \"92323125-b9ec-4231-beb8-180abd22ab0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:42:41.677436 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.677410    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck7ph\" (UniqueName: \"kubernetes.io/projected/92323125-b9ec-4231-beb8-180abd22ab0f-kube-api-access-ck7ph\") pod \"kuadrant-operator-controller-manager-57dd85bb4-nclhb\" (UID: \"92323125-b9ec-4231-beb8-180abd22ab0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:42:41.816318 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.816287    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:42:41.953240 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:41.953207    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"]
May 20 15:42:41.957551 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:42:41.957527    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92323125_b9ec_4231_beb8_180abd22ab0f.slice/crio-7dbdf5dd99172d017689286b52fc819860125f1eff71701edbc6784b52af7bae WatchSource:0}: Error finding container 7dbdf5dd99172d017689286b52fc819860125f1eff71701edbc6784b52af7bae: Status 404 returned error can't find the container with id 7dbdf5dd99172d017689286b52fc819860125f1eff71701edbc6784b52af7bae
May 20 15:42:42.119621 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:42.119535    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb" event={"ID":"92323125-b9ec-4231-beb8-180abd22ab0f","Type":"ContainerStarted","Data":"7dbdf5dd99172d017689286b52fc819860125f1eff71701edbc6784b52af7bae"}
May 20 15:42:47.138849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:47.138734    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb" event={"ID":"92323125-b9ec-4231-beb8-180abd22ab0f","Type":"ContainerStarted","Data":"20b532d7ecac23386a65dcc35aa56abf40255b2f0ea006cbf09d8aa6be6aa31b"}
May 20 15:42:47.138849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:47.138809    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:42:47.194518 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:47.194469    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb" podStartSLOduration=1.2898852060000001 podStartE2EDuration="6.194454305s" podCreationTimestamp="2026-05-20 15:42:41 +0000 UTC" firstStartedPulling="2026-05-20 15:42:41.959942035 +0000 UTC m=+671.643811351" lastFinishedPulling="2026-05-20 15:42:46.864511129 +0000 UTC m=+676.548380450" observedRunningTime="2026-05-20 15:42:47.19199 +0000 UTC m=+676.875859339" watchObservedRunningTime="2026-05-20 15:42:47.194454305 +0000 UTC m=+676.878323643"
May 20 15:42:48.108132 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:48.108094    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-tt77j"
May 20 15:42:58.145137 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:42:58.145101    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-nclhb"
May 20 15:43:30.461787 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.461681    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-674b59b84c-sfwqz"]
May 20 15:43:30.466537 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.466512    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-sfwqz"
May 20 15:43:30.469929 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.469907    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-s5dc5\""
May 20 15:43:30.474974 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.474950    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sfwqz"]
May 20 15:43:30.494722 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.494697    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pwkc\" (UniqueName: \"kubernetes.io/projected/269e888e-faee-4f59-b4ba-703df97bd5c9-kube-api-access-2pwkc\") pod \"authorino-674b59b84c-sfwqz\" (UID: \"269e888e-faee-4f59-b4ba-703df97bd5c9\") " pod="kuadrant-system/authorino-674b59b84c-sfwqz"
May 20 15:43:30.595817 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.595782    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwkc\" (UniqueName: \"kubernetes.io/projected/269e888e-faee-4f59-b4ba-703df97bd5c9-kube-api-access-2pwkc\") pod \"authorino-674b59b84c-sfwqz\" (UID: \"269e888e-faee-4f59-b4ba-703df97bd5c9\") " pod="kuadrant-system/authorino-674b59b84c-sfwqz"
May 20 15:43:30.606258 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.606235    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pwkc\" (UniqueName: \"kubernetes.io/projected/269e888e-faee-4f59-b4ba-703df97bd5c9-kube-api-access-2pwkc\") pod \"authorino-674b59b84c-sfwqz\" (UID: \"269e888e-faee-4f59-b4ba-703df97bd5c9\") " pod="kuadrant-system/authorino-674b59b84c-sfwqz"
May 20 15:43:30.691849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.691819    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79cbc94b89-87s5v"]
May 20 15:43:30.694209 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.694186    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-87s5v"
May 20 15:43:30.704890 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.704865    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-87s5v"]
May 20 15:43:30.776226 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.776191    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-sfwqz"
May 20 15:43:30.797229 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.797204    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c74g\" (UniqueName: \"kubernetes.io/projected/75e69545-6a37-4087-9cf4-49d66122fa6d-kube-api-access-9c74g\") pod \"authorino-79cbc94b89-87s5v\" (UID: \"75e69545-6a37-4087-9cf4-49d66122fa6d\") " pod="kuadrant-system/authorino-79cbc94b89-87s5v"
May 20 15:43:30.898241 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.898203    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9c74g\" (UniqueName: \"kubernetes.io/projected/75e69545-6a37-4087-9cf4-49d66122fa6d-kube-api-access-9c74g\") pod \"authorino-79cbc94b89-87s5v\" (UID: \"75e69545-6a37-4087-9cf4-49d66122fa6d\") " pod="kuadrant-system/authorino-79cbc94b89-87s5v"
May 20 15:43:30.909952 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.909931    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sfwqz"]
May 20 15:43:30.910779 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:30.910740    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c74g\" (UniqueName: \"kubernetes.io/projected/75e69545-6a37-4087-9cf4-49d66122fa6d-kube-api-access-9c74g\") pod \"authorino-79cbc94b89-87s5v\" (UID: \"75e69545-6a37-4087-9cf4-49d66122fa6d\") " pod="kuadrant-system/authorino-79cbc94b89-87s5v"
May 20 15:43:30.912474 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:43:30.912441    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod269e888e_faee_4f59_b4ba_703df97bd5c9.slice/crio-56a360e7b5de5c1238d2aba78e546d9ab3ad4158a2425d643a15d63a8c09ee66 WatchSource:0}: Error finding container 56a360e7b5de5c1238d2aba78e546d9ab3ad4158a2425d643a15d63a8c09ee66: Status 404 returned error can't find the container with id 56a360e7b5de5c1238d2aba78e546d9ab3ad4158a2425d643a15d63a8c09ee66
May 20 15:43:31.004225 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:31.004193    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-87s5v"
May 20 15:43:31.133501 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:31.133470    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-87s5v"]
May 20 15:43:31.136773 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:43:31.136729    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75e69545_6a37_4087_9cf4_49d66122fa6d.slice/crio-e6831ff6d76c94f950af1c7ca0f1fcada153eb2f79f0a7fcdfbb83f44a21c685 WatchSource:0}: Error finding container e6831ff6d76c94f950af1c7ca0f1fcada153eb2f79f0a7fcdfbb83f44a21c685: Status 404 returned error can't find the container with id e6831ff6d76c94f950af1c7ca0f1fcada153eb2f79f0a7fcdfbb83f44a21c685
May 20 15:43:31.281370 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:31.281328    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-sfwqz" event={"ID":"269e888e-faee-4f59-b4ba-703df97bd5c9","Type":"ContainerStarted","Data":"56a360e7b5de5c1238d2aba78e546d9ab3ad4158a2425d643a15d63a8c09ee66"}
May 20 15:43:31.282411 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:31.282386    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-87s5v" event={"ID":"75e69545-6a37-4087-9cf4-49d66122fa6d","Type":"ContainerStarted","Data":"e6831ff6d76c94f950af1c7ca0f1fcada153eb2f79f0a7fcdfbb83f44a21c685"}
May 20 15:43:33.292003 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:33.291963    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-sfwqz" event={"ID":"269e888e-faee-4f59-b4ba-703df97bd5c9","Type":"ContainerStarted","Data":"9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd"}
May 20 15:43:33.293316 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:33.293291    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-87s5v" event={"ID":"75e69545-6a37-4087-9cf4-49d66122fa6d","Type":"ContainerStarted","Data":"1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb"}
May 20 15:43:33.311363 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:33.311319    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-674b59b84c-sfwqz" podStartSLOduration=1.730257093 podStartE2EDuration="3.31130731s" podCreationTimestamp="2026-05-20 15:43:30 +0000 UTC" firstStartedPulling="2026-05-20 15:43:30.913865539 +0000 UTC m=+720.597734856" lastFinishedPulling="2026-05-20 15:43:32.494915745 +0000 UTC m=+722.178785073" observedRunningTime="2026-05-20 15:43:33.310304418 +0000 UTC m=+722.994173756" watchObservedRunningTime="2026-05-20 15:43:33.31130731 +0000 UTC m=+722.995176649"
May 20 15:43:33.333418 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:33.333374    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-79cbc94b89-87s5v" podStartSLOduration=1.924078598 podStartE2EDuration="3.333361973s" podCreationTimestamp="2026-05-20 15:43:30 +0000 UTC" firstStartedPulling="2026-05-20 15:43:31.138045958 +0000 UTC m=+720.821915279" lastFinishedPulling="2026-05-20 15:43:32.547329317 +0000 UTC m=+722.231198654" observedRunningTime="2026-05-20 15:43:33.329568187 +0000 UTC m=+723.013437526" watchObservedRunningTime="2026-05-20 15:43:33.333361973 +0000 UTC m=+723.017231312"
May 20 15:43:33.373340 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:33.373307    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sfwqz"]
May 20 15:43:35.300928 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:35.300885    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-674b59b84c-sfwqz" podUID="269e888e-faee-4f59-b4ba-703df97bd5c9" containerName="authorino" containerID="cri-o://9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd" gracePeriod=30
May 20 15:43:35.548027 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:35.548003    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-sfwqz"
May 20 15:43:35.643690 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:35.643597    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pwkc\" (UniqueName: \"kubernetes.io/projected/269e888e-faee-4f59-b4ba-703df97bd5c9-kube-api-access-2pwkc\") pod \"269e888e-faee-4f59-b4ba-703df97bd5c9\" (UID: \"269e888e-faee-4f59-b4ba-703df97bd5c9\") "
May 20 15:43:35.645931 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:35.645903    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/269e888e-faee-4f59-b4ba-703df97bd5c9-kube-api-access-2pwkc" (OuterVolumeSpecName: "kube-api-access-2pwkc") pod "269e888e-faee-4f59-b4ba-703df97bd5c9" (UID: "269e888e-faee-4f59-b4ba-703df97bd5c9"). InnerVolumeSpecName "kube-api-access-2pwkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:43:35.745042 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:35.745011    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pwkc\" (UniqueName: \"kubernetes.io/projected/269e888e-faee-4f59-b4ba-703df97bd5c9-kube-api-access-2pwkc\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:43:36.305493 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.305457    2566 generic.go:358] "Generic (PLEG): container finished" podID="269e888e-faee-4f59-b4ba-703df97bd5c9" containerID="9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd" exitCode=0
May 20 15:43:36.305996 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.305509    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674b59b84c-sfwqz"
May 20 15:43:36.305996 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.305547    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-sfwqz" event={"ID":"269e888e-faee-4f59-b4ba-703df97bd5c9","Type":"ContainerDied","Data":"9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd"}
May 20 15:43:36.305996 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.305587    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674b59b84c-sfwqz" event={"ID":"269e888e-faee-4f59-b4ba-703df97bd5c9","Type":"ContainerDied","Data":"56a360e7b5de5c1238d2aba78e546d9ab3ad4158a2425d643a15d63a8c09ee66"}
May 20 15:43:36.305996 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.305613    2566 scope.go:117] "RemoveContainer" containerID="9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd"
May 20 15:43:36.316236 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.316218    2566 scope.go:117] "RemoveContainer" containerID="9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd"
May 20 15:43:36.316528 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:43:36.316511    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd\": container with ID starting with 9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd not found: ID does not exist" containerID="9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd"
May 20 15:43:36.316582 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.316539    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd"} err="failed to get container status \"9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd\": rpc error: code = NotFound desc = could not find container \"9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd\": container with ID starting with 9ca32fcc74ae6bf29984f3547a43484876efb54b9d587a39001c7ee5e2efa4fd not found: ID does not exist"
May 20 15:43:36.329892 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.329870    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sfwqz"]
May 20 15:43:36.335595 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.335567    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-674b59b84c-sfwqz"]
May 20 15:43:36.932598 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:36.932562    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="269e888e-faee-4f59-b4ba-703df97bd5c9" path="/var/lib/kubelet/pods/269e888e-faee-4f59-b4ba-703df97bd5c9/volumes"
May 20 15:43:58.807919 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:58.807883    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68bd676465-j4s95"]
May 20 15:43:58.808440 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:58.808307    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="269e888e-faee-4f59-b4ba-703df97bd5c9" containerName="authorino"
May 20 15:43:58.808440 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:58.808326    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="269e888e-faee-4f59-b4ba-703df97bd5c9" containerName="authorino"
May 20 15:43:58.808440 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:58.808397    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="269e888e-faee-4f59-b4ba-703df97bd5c9" containerName="authorino"
May 20 15:43:58.810536 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:58.810520    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-j4s95"
May 20 15:43:58.813560 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:58.813542    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
May 20 15:43:58.828359 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:58.828335    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-j4s95"]
May 20 15:43:58.938253 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:58.938226    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6a8c445-187a-4c3b-82f3-8d58e7d53436-tls-cert\") pod \"authorino-68bd676465-j4s95\" (UID: \"a6a8c445-187a-4c3b-82f3-8d58e7d53436\") " pod="kuadrant-system/authorino-68bd676465-j4s95"
May 20 15:43:58.938425 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:58.938273    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crngm\" (UniqueName: \"kubernetes.io/projected/a6a8c445-187a-4c3b-82f3-8d58e7d53436-kube-api-access-crngm\") pod \"authorino-68bd676465-j4s95\" (UID: \"a6a8c445-187a-4c3b-82f3-8d58e7d53436\") " pod="kuadrant-system/authorino-68bd676465-j4s95"
May 20 15:43:59.039578 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:59.039536    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-crngm\" (UniqueName: \"kubernetes.io/projected/a6a8c445-187a-4c3b-82f3-8d58e7d53436-kube-api-access-crngm\") pod \"authorino-68bd676465-j4s95\" (UID: \"a6a8c445-187a-4c3b-82f3-8d58e7d53436\") " pod="kuadrant-system/authorino-68bd676465-j4s95"
May 20 15:43:59.039826 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:59.039632    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6a8c445-187a-4c3b-82f3-8d58e7d53436-tls-cert\") pod \"authorino-68bd676465-j4s95\" (UID: \"a6a8c445-187a-4c3b-82f3-8d58e7d53436\") " pod="kuadrant-system/authorino-68bd676465-j4s95"
May 20 15:43:59.042374 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:59.042348    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6a8c445-187a-4c3b-82f3-8d58e7d53436-tls-cert\") pod \"authorino-68bd676465-j4s95\" (UID: \"a6a8c445-187a-4c3b-82f3-8d58e7d53436\") " pod="kuadrant-system/authorino-68bd676465-j4s95"
May 20 15:43:59.052438 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:59.052415    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crngm\" (UniqueName: \"kubernetes.io/projected/a6a8c445-187a-4c3b-82f3-8d58e7d53436-kube-api-access-crngm\") pod \"authorino-68bd676465-j4s95\" (UID: \"a6a8c445-187a-4c3b-82f3-8d58e7d53436\") " pod="kuadrant-system/authorino-68bd676465-j4s95"
May 20 15:43:59.119489 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:59.119405    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68bd676465-j4s95"
May 20 15:43:59.266334 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:59.266305    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68bd676465-j4s95"]
May 20 15:43:59.270172 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:43:59.270145    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6a8c445_187a_4c3b_82f3_8d58e7d53436.slice/crio-df69dd7376d9f7117491ed869f6c11ece50c7ad0fc57b99207bbbc6297c0a3af WatchSource:0}: Error finding container df69dd7376d9f7117491ed869f6c11ece50c7ad0fc57b99207bbbc6297c0a3af: Status 404 returned error can't find the container with id df69dd7376d9f7117491ed869f6c11ece50c7ad0fc57b99207bbbc6297c0a3af
May 20 15:43:59.384058 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:43:59.383963    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-j4s95" event={"ID":"a6a8c445-187a-4c3b-82f3-8d58e7d53436","Type":"ContainerStarted","Data":"df69dd7376d9f7117491ed869f6c11ece50c7ad0fc57b99207bbbc6297c0a3af"}
May 20 15:44:00.388620 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:00.388586    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68bd676465-j4s95" event={"ID":"a6a8c445-187a-4c3b-82f3-8d58e7d53436","Type":"ContainerStarted","Data":"8b53f951c5942674d06da0990c1b5d9a2a90b00a525d8aeb41feda368878365b"}
May 20 15:44:00.413889 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:00.413833    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-68bd676465-j4s95" podStartSLOduration=1.708460383 podStartE2EDuration="2.413817515s" podCreationTimestamp="2026-05-20 15:43:58 +0000 UTC" firstStartedPulling="2026-05-20 15:43:59.271875378 +0000 UTC m=+748.955744694" lastFinishedPulling="2026-05-20 15:43:59.977232506 +0000 UTC m=+749.661101826" observedRunningTime="2026-05-20 15:44:00.412484579 +0000 UTC m=+750.096353920" watchObservedRunningTime="2026-05-20 15:44:00.413817515 +0000 UTC m=+750.097686853"
May 20 15:44:00.469360 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:00.469329    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-87s5v"]
May 20 15:44:00.469542 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:00.469522    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-79cbc94b89-87s5v" podUID="75e69545-6a37-4087-9cf4-49d66122fa6d" containerName="authorino" containerID="cri-o://1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb" gracePeriod=30
May 20 15:44:00.706025 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:00.706004    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-87s5v"
May 20 15:44:00.853297 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:00.853255    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c74g\" (UniqueName: \"kubernetes.io/projected/75e69545-6a37-4087-9cf4-49d66122fa6d-kube-api-access-9c74g\") pod \"75e69545-6a37-4087-9cf4-49d66122fa6d\" (UID: \"75e69545-6a37-4087-9cf4-49d66122fa6d\") "
May 20 15:44:00.855499 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:00.855460    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e69545-6a37-4087-9cf4-49d66122fa6d-kube-api-access-9c74g" (OuterVolumeSpecName: "kube-api-access-9c74g") pod "75e69545-6a37-4087-9cf4-49d66122fa6d" (UID: "75e69545-6a37-4087-9cf4-49d66122fa6d"). InnerVolumeSpecName "kube-api-access-9c74g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:44:00.953920 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:00.953858    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c74g\" (UniqueName: \"kubernetes.io/projected/75e69545-6a37-4087-9cf4-49d66122fa6d-kube-api-access-9c74g\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:44:01.392710 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:01.392671    2566 generic.go:358] "Generic (PLEG): container finished" podID="75e69545-6a37-4087-9cf4-49d66122fa6d" containerID="1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb" exitCode=0
May 20 15:44:01.393135 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:01.392724    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-87s5v"
May 20 15:44:01.393135 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:01.392762    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-87s5v" event={"ID":"75e69545-6a37-4087-9cf4-49d66122fa6d","Type":"ContainerDied","Data":"1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb"}
May 20 15:44:01.393135 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:01.392814    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-87s5v" event={"ID":"75e69545-6a37-4087-9cf4-49d66122fa6d","Type":"ContainerDied","Data":"e6831ff6d76c94f950af1c7ca0f1fcada153eb2f79f0a7fcdfbb83f44a21c685"}
May 20 15:44:01.393135 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:01.392839    2566 scope.go:117] "RemoveContainer" containerID="1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb"
May 20 15:44:01.401331 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:01.401313    2566 scope.go:117] "RemoveContainer" containerID="1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb"
May 20 15:44:01.401582 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:44:01.401563    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb\": container with ID starting with 1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb not found: ID does not exist" containerID="1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb"
May 20 15:44:01.401652 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:01.401593    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb"} err="failed to get container status \"1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb\": rpc error: code = NotFound desc = could not find container \"1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb\": container with ID starting with 1897501dc146b72be9bfc8edf9abcc3f6d0a34bcf8fdbb275cc5205fbc3e2eeb not found: ID does not exist"
May 20 15:44:01.412699 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:01.412678    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-87s5v"]
May 20 15:44:01.416665 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:01.416647    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-87s5v"]
May 20 15:44:02.932292 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:02.932262    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e69545-6a37-4087-9cf4-49d66122fa6d" path="/var/lib/kubelet/pods/75e69545-6a37-4087-9cf4-49d66122fa6d/volumes"
May 20 15:44:38.062161 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.062130    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-86f7b4b499-5tt55"]
May 20 15:44:38.062696 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.062595    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e69545-6a37-4087-9cf4-49d66122fa6d" containerName="authorino"
May 20 15:44:38.062696 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.062613    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e69545-6a37-4087-9cf4-49d66122fa6d" containerName="authorino"
May 20 15:44:38.062862 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.062706    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="75e69545-6a37-4087-9cf4-49d66122fa6d" containerName="authorino"
May 20 15:44:38.065838 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.065812    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:44:38.068324 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.068303    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
May 20 15:44:38.069152 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.069134    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
May 20 15:44:38.069253 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.069157    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
May 20 15:44:38.069253 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.069186    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-brcl5\""
May 20 15:44:38.081127 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.081104    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-5tt55"]
May 20 15:44:38.162507 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.162479    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwfl9\" (UniqueName: \"kubernetes.io/projected/41b6fa64-2b81-489b-91a8-65dd860aac80-kube-api-access-zwfl9\") pod \"model-serving-api-86f7b4b499-5tt55\" (UID: \"41b6fa64-2b81-489b-91a8-65dd860aac80\") " pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:44:38.162684 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.162526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41b6fa64-2b81-489b-91a8-65dd860aac80-tls-certs\") pod \"model-serving-api-86f7b4b499-5tt55\" (UID: \"41b6fa64-2b81-489b-91a8-65dd860aac80\") " pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:44:38.263853 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.263820    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zwfl9\" (UniqueName: \"kubernetes.io/projected/41b6fa64-2b81-489b-91a8-65dd860aac80-kube-api-access-zwfl9\") pod \"model-serving-api-86f7b4b499-5tt55\" (UID: \"41b6fa64-2b81-489b-91a8-65dd860aac80\") " pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:44:38.264055 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.263869    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41b6fa64-2b81-489b-91a8-65dd860aac80-tls-certs\") pod \"model-serving-api-86f7b4b499-5tt55\" (UID: \"41b6fa64-2b81-489b-91a8-65dd860aac80\") " pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:44:38.266398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.266371    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41b6fa64-2b81-489b-91a8-65dd860aac80-tls-certs\") pod \"model-serving-api-86f7b4b499-5tt55\" (UID: \"41b6fa64-2b81-489b-91a8-65dd860aac80\") " pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:44:38.273918 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.273894    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwfl9\" (UniqueName: \"kubernetes.io/projected/41b6fa64-2b81-489b-91a8-65dd860aac80-kube-api-access-zwfl9\") pod \"model-serving-api-86f7b4b499-5tt55\" (UID: \"41b6fa64-2b81-489b-91a8-65dd860aac80\") " pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:44:38.375463 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.375368    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:44:38.504659 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.504627    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-5tt55"]
May 20 15:44:38.507912 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:44:38.507881    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b6fa64_2b81_489b_91a8_65dd860aac80.slice/crio-6fdeb1a3a150d08197ea89173f60a8a75d39ca89e5eb6ec12dfc0c400f1026ca WatchSource:0}: Error finding container 6fdeb1a3a150d08197ea89173f60a8a75d39ca89e5eb6ec12dfc0c400f1026ca: Status 404 returned error can't find the container with id 6fdeb1a3a150d08197ea89173f60a8a75d39ca89e5eb6ec12dfc0c400f1026ca
May 20 15:44:38.520898 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:38.520871    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-5tt55" event={"ID":"41b6fa64-2b81-489b-91a8-65dd860aac80","Type":"ContainerStarted","Data":"6fdeb1a3a150d08197ea89173f60a8a75d39ca89e5eb6ec12dfc0c400f1026ca"}
May 20 15:44:41.533637 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:41.533600    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-5tt55" event={"ID":"41b6fa64-2b81-489b-91a8-65dd860aac80","Type":"ContainerStarted","Data":"84777e682c8032349be47dc807e1d1cba82e5649473dd0d44d6f4983dfdceb30"}
May 20 15:44:41.534033 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:41.533729    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:44:41.551990 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:41.551937    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/model-serving-api-86f7b4b499-5tt55" podStartSLOduration=1.444835912 podStartE2EDuration="3.551923139s" podCreationTimestamp="2026-05-20 15:44:38 +0000 UTC" firstStartedPulling="2026-05-20 15:44:38.510039126 +0000 UTC m=+788.193908443" lastFinishedPulling="2026-05-20 15:44:40.617126338 +0000 UTC m=+790.300995670" observedRunningTime="2026-05-20 15:44:41.550871186 +0000 UTC m=+791.234740525" watchObservedRunningTime="2026-05-20 15:44:41.551923139 +0000 UTC m=+791.235792478"
May 20 15:44:52.541268 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:44:52.541236    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-86f7b4b499-5tt55"
May 20 15:45:29.544476 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.544433    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"]
May 20 15:45:29.547857 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.547837    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.551604 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.551580    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
May 20 15:45:29.551604 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.551586    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sxlvb\""
May 20 15:45:29.551851 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.551610    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
May 20 15:45:29.551851 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.551586    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
May 20 15:45:29.559650 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.559623    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"]
May 20 15:45:29.591502 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.591475    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmfpr\" (UniqueName: \"kubernetes.io/projected/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kube-api-access-fmfpr\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.591634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.591509    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-home\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.591634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.591530    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tmp-dir\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.591634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.591549    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tls-certs\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.591634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.591592    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.591634 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.591621    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-model-cache\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.591871 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.591704    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-dshm\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.692873 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.692837    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tls-certs\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.692873 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.692874    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.693095 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.692895    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-model-cache\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.693095 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.692968    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-dshm\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.693095 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.693022    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fmfpr\" (UniqueName: \"kubernetes.io/projected/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kube-api-access-fmfpr\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.693095 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.693039    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-home\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.693095 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.693060    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tmp-dir\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.693341 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.693304    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.693341 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.693320    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-model-cache\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.693454 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.693389    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tmp-dir\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.693454 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.693443    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-home\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.695437 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.695413    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-dshm\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.695629 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.695614    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tls-certs\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.701175 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.701152    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmfpr\" (UniqueName: \"kubernetes.io/projected/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kube-api-access-fmfpr\") pod \"scheduler-inline-config-test-kserve-6f8d85b996-zmszg\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.858880 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.858776    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:45:29.994904 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:29.994863    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"]
May 20 15:45:29.998072 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:45:29.998040    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd56ea84c_ac0d_43d7_bc12_516a1f10c2b9.slice/crio-82c5c90fb4488d46f7bd1cdc9f371965c4992ffe760f203f9757aba206aa945f WatchSource:0}: Error finding container 82c5c90fb4488d46f7bd1cdc9f371965c4992ffe760f203f9757aba206aa945f: Status 404 returned error can't find the container with id 82c5c90fb4488d46f7bd1cdc9f371965c4992ffe760f203f9757aba206aa945f
May 20 15:45:30.694400 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:30.694362    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg" event={"ID":"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9","Type":"ContainerStarted","Data":"82c5c90fb4488d46f7bd1cdc9f371965c4992ffe760f203f9757aba206aa945f"}
May 20 15:45:33.706957 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:45:33.706875    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg" event={"ID":"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9","Type":"ContainerStarted","Data":"1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d"}
May 20 15:46:30.887716 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:46:30.887638    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:46:30.889719 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:46:30.889689    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:46:30.894280 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:46:30.894261    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:46:30.896000 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:46:30.895840    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:47:33.098141 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:33.098101    2566 generic.go:358] "Generic (PLEG): container finished" podID="d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" containerID="1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d" exitCode=0
May 20 15:47:33.098725 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:33.098139    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg" event={"ID":"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9","Type":"ContainerDied","Data":"1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d"}
May 20 15:47:33.099220 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:33.099201    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 20 15:47:35.110012 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:35.109977    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg" event={"ID":"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9","Type":"ContainerStarted","Data":"a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace"}
May 20 15:47:35.135105 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:35.130184    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg" podStartSLOduration=1.659025245 podStartE2EDuration="2m6.130165962s" podCreationTimestamp="2026-05-20 15:45:29 +0000 UTC" firstStartedPulling="2026-05-20 15:45:29.999980249 +0000 UTC m=+839.683849569" lastFinishedPulling="2026-05-20 15:47:34.471120955 +0000 UTC m=+964.154990286" observedRunningTime="2026-05-20 15:47:35.12808797 +0000 UTC m=+964.811957310" watchObservedRunningTime="2026-05-20 15:47:35.130165962 +0000 UTC m=+964.814035302"
May 20 15:47:39.859204 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:39.859152    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:47:39.859204 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:39.859218    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:47:39.872969 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:39.872944    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:47:40.138698 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:40.138616    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:47:41.859257 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:41.859174    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"]
May 20 15:47:42.134097 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.133970    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg" podUID="d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" containerName="main" containerID="cri-o://a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace" gracePeriod=30
May 20 15:47:42.398847 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.398825    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:47:42.544355 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544326    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-model-cache\") pod \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") "
May 20 15:47:42.544553 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544390    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kserve-provision-location\") pod \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") "
May 20 15:47:42.544553 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544440    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-dshm\") pod \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") "
May 20 15:47:42.544553 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544483    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmfpr\" (UniqueName: \"kubernetes.io/projected/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kube-api-access-fmfpr\") pod \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") "
May 20 15:47:42.544553 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544505    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-home\") pod \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") "
May 20 15:47:42.544553 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544543    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tls-certs\") pod \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") "
May 20 15:47:42.544846 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544576    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tmp-dir\") pod \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\" (UID: \"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9\") "
May 20 15:47:42.544904 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544856    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-home" (OuterVolumeSpecName: "home") pod "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" (UID: "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:47:42.544966 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544889    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" (UID: "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:47:42.544966 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.544935    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-model-cache" (OuterVolumeSpecName: "model-cache") pod "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" (UID: "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:47:42.546963 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.546930    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kube-api-access-fmfpr" (OuterVolumeSpecName: "kube-api-access-fmfpr") pod "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" (UID: "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9"). InnerVolumeSpecName "kube-api-access-fmfpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:47:42.547068 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.546965    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" (UID: "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:47:42.547068 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.547023    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-dshm" (OuterVolumeSpecName: "dshm") pod "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" (UID: "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:47:42.604884 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.604838    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" (UID: "d56ea84c-ac0d-43d7-bc12-516a1f10c2b9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:47:42.645294 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.645214    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kserve-provision-location\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:47:42.645294 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.645238    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-dshm\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:47:42.645294 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.645250    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmfpr\" (UniqueName: \"kubernetes.io/projected/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-kube-api-access-fmfpr\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:47:42.645294 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.645261    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-home\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:47:42.645294 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.645271    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tls-certs\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:47:42.645294 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.645282    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-tmp-dir\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:47:42.645294 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:42.645289    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9-model-cache\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:47:43.138816 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.138780    2566 generic.go:358] "Generic (PLEG): container finished" podID="d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" containerID="a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace" exitCode=0
May 20 15:47:43.139284 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.138860    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"
May 20 15:47:43.139284 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.138870    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg" event={"ID":"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9","Type":"ContainerDied","Data":"a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace"}
May 20 15:47:43.139284 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.138919    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg" event={"ID":"d56ea84c-ac0d-43d7-bc12-516a1f10c2b9","Type":"ContainerDied","Data":"82c5c90fb4488d46f7bd1cdc9f371965c4992ffe760f203f9757aba206aa945f"}
May 20 15:47:43.139284 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.138942    2566 scope.go:117] "RemoveContainer" containerID="a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace"
May 20 15:47:43.147398 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.147355    2566 scope.go:117] "RemoveContainer" containerID="1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d"
May 20 15:47:43.167043 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.167014    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"]
May 20 15:47:43.175034 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.175010    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-6f8d85b996-zmszg"]
May 20 15:47:43.212695 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.212672    2566 scope.go:117] "RemoveContainer" containerID="a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace"
May 20 15:47:43.213074 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:47:43.213051    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace\": container with ID starting with a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace not found: ID does not exist" containerID="a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace"
May 20 15:47:43.213169 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.213081    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace"} err="failed to get container status \"a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace\": rpc error: code = NotFound desc = could not find container \"a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace\": container with ID starting with a5a189c137aae715eeb2e76663a9bf078badda46888fb39656ef498dd5f35ace not found: ID does not exist"
May 20 15:47:43.213169 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.213103    2566 scope.go:117] "RemoveContainer" containerID="1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d"
May 20 15:47:43.213399 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:47:43.213382    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d\": container with ID starting with 1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d not found: ID does not exist" containerID="1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d"
May 20 15:47:43.213444 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:43.213406    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d"} err="failed to get container status \"1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d\": rpc error: code = NotFound desc = could not find container \"1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d\": container with ID starting with 1561ccd87f187d950117e384955b6823b536a69f8390f7f259a385bae8e2a20d not found: ID does not exist"
May 20 15:47:44.933536 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:44.933500    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" path="/var/lib/kubelet/pods/d56ea84c-ac0d-43d7-bc12-516a1f10c2b9/volumes"
May 20 15:47:49.445626 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.445590    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"]
May 20 15:47:49.446149 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.446114    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" containerName="storage-initializer"
May 20 15:47:49.446149 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.446133    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" containerName="storage-initializer"
May 20 15:47:49.446149 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.446147    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" containerName="main"
May 20 15:47:49.446249 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.446156    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" containerName="main"
May 20 15:47:49.446249 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.446231    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="d56ea84c-ac0d-43d7-bc12-516a1f10c2b9" containerName="main"
May 20 15:47:49.448755 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.448725    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.451390 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.451370    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
May 20 15:47:49.452514 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.452487    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sxlvb\""
May 20 15:47:49.452625 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.452488    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
May 20 15:47:49.452625 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.452541    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
May 20 15:47:49.460088 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.460065    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"]
May 20 15:47:49.502815 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.502777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn8pg\" (UniqueName: \"kubernetes.io/projected/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kube-api-access-cn8pg\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.502951 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.502827    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-dshm\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.502951 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.502880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.502951 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.502909    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-home\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.502951 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.502927    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.502951 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.502947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.503116 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.503024    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.603692 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.603658    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cn8pg\" (UniqueName: \"kubernetes.io/projected/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kube-api-access-cn8pg\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.603692 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.603695    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-dshm\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.603923 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.603721    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.603923 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.603807    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-home\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.603923 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.603851    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.604077 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.603957    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.604077 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.604047    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.604191 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.604149    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.604235 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.604185    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-home\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.604346 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.604314    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.604501 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.604481    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.606339 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.606316    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-dshm\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.606451 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.606434    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.611841 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.611814    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn8pg\" (UniqueName: \"kubernetes.io/projected/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kube-api-access-cn8pg\") pod \"scheduler-configmap-ref-test-kserve-5d5597645b-99r2f\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.760585 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.760493    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:47:49.904742 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:49.904710    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"]
May 20 15:47:49.905390 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:47:49.905366    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd4cf93_a476_49c4_8f52_5f0b823d530a.slice/crio-47e40de1eaa8720ef3695d5be4130cf0bcba5c965fc4b0deff1392719ad7e700 WatchSource:0}: Error finding container 47e40de1eaa8720ef3695d5be4130cf0bcba5c965fc4b0deff1392719ad7e700: Status 404 returned error can't find the container with id 47e40de1eaa8720ef3695d5be4130cf0bcba5c965fc4b0deff1392719ad7e700
May 20 15:47:50.167089 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:50.167004    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" event={"ID":"7cd4cf93-a476-49c4-8f52-5f0b823d530a","Type":"ContainerStarted","Data":"e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713"}
May 20 15:47:50.167089 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:47:50.167039    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" event={"ID":"7cd4cf93-a476-49c4-8f52-5f0b823d530a","Type":"ContainerStarted","Data":"47e40de1eaa8720ef3695d5be4130cf0bcba5c965fc4b0deff1392719ad7e700"}
May 20 15:49:07.426011 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:07.425975    2566 generic.go:358] "Generic (PLEG): container finished" podID="7cd4cf93-a476-49c4-8f52-5f0b823d530a" containerID="e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713" exitCode=0
May 20 15:49:07.426415 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:07.426057    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" event={"ID":"7cd4cf93-a476-49c4-8f52-5f0b823d530a","Type":"ContainerDied","Data":"e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713"}
May 20 15:49:08.431307 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:08.431269    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" event={"ID":"7cd4cf93-a476-49c4-8f52-5f0b823d530a","Type":"ContainerStarted","Data":"b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c"}
May 20 15:49:08.453112 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:08.453059    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" podStartSLOduration=79.453042468 podStartE2EDuration="1m19.453042468s" podCreationTimestamp="2026-05-20 15:47:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:49:08.450988694 +0000 UTC m=+1058.134858035" watchObservedRunningTime="2026-05-20 15:49:08.453042468 +0000 UTC m=+1058.136911808"
May 20 15:49:09.761500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:09.761466    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:49:09.761500 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:09.761509    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:49:09.774013 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:09.773986    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:49:10.449160 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:10.449131    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:49:11.471850 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:11.471812    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"]
May 20 15:49:11.615109 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:49:11.615069    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs: secret "scheduler-configmap-ref-test-kserve-self-signed-certs" not found
May 20 15:49:11.615322 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:49:11.615162    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs podName:7cd4cf93-a476-49c4-8f52-5f0b823d530a nodeName:}" failed. No retries permitted until 2026-05-20 15:49:12.115139435 +0000 UTC m=+1061.799008753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs") pod "scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" (UID: "7cd4cf93-a476-49c4-8f52-5f0b823d530a") : secret "scheduler-configmap-ref-test-kserve-self-signed-certs" not found
May 20 15:49:12.120333 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:49:12.120297    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs: secret "scheduler-configmap-ref-test-kserve-self-signed-certs" not found
May 20 15:49:12.120527 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:49:12.120370    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs podName:7cd4cf93-a476-49c4-8f52-5f0b823d530a nodeName:}" failed. No retries permitted until 2026-05-20 15:49:13.120353968 +0000 UTC m=+1062.804223285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs") pod "scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" (UID: "7cd4cf93-a476-49c4-8f52-5f0b823d530a") : secret "scheduler-configmap-ref-test-kserve-self-signed-certs" not found
May 20 15:49:12.446530 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.446423    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" podUID="7cd4cf93-a476-49c4-8f52-5f0b823d530a" containerName="main" containerID="cri-o://b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c" gracePeriod=30
May 20 15:49:12.695857 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.695835    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:49:12.825861 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.825823    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs\") pod \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") "
May 20 15:49:12.826045 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.825887    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tmp-dir\") pod \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") "
May 20 15:49:12.826045 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.825919    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-model-cache\") pod \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") "
May 20 15:49:12.826045 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.825955    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-home\") pod \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") "
May 20 15:49:12.826045 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.825998    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-dshm\") pod \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") "
May 20 15:49:12.826045 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.826028    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn8pg\" (UniqueName: \"kubernetes.io/projected/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kube-api-access-cn8pg\") pod \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") "
May 20 15:49:12.826309 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.826060    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kserve-provision-location\") pod \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\" (UID: \"7cd4cf93-a476-49c4-8f52-5f0b823d530a\") "
May 20 15:49:12.826309 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.826192    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-model-cache" (OuterVolumeSpecName: "model-cache") pod "7cd4cf93-a476-49c4-8f52-5f0b823d530a" (UID: "7cd4cf93-a476-49c4-8f52-5f0b823d530a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:49:12.826309 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.826263    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-home" (OuterVolumeSpecName: "home") pod "7cd4cf93-a476-49c4-8f52-5f0b823d530a" (UID: "7cd4cf93-a476-49c4-8f52-5f0b823d530a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:49:12.826456 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.826333    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7cd4cf93-a476-49c4-8f52-5f0b823d530a" (UID: "7cd4cf93-a476-49c4-8f52-5f0b823d530a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:49:12.826456 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.826434    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tmp-dir\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:49:12.826456 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.826452    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-model-cache\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:49:12.826623 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.826467    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-home\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:49:12.828444 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.828407    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7cd4cf93-a476-49c4-8f52-5f0b823d530a" (UID: "7cd4cf93-a476-49c4-8f52-5f0b823d530a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:49:12.828847 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.828817    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-dshm" (OuterVolumeSpecName: "dshm") pod "7cd4cf93-a476-49c4-8f52-5f0b823d530a" (UID: "7cd4cf93-a476-49c4-8f52-5f0b823d530a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:49:12.828955 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.828905    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kube-api-access-cn8pg" (OuterVolumeSpecName: "kube-api-access-cn8pg") pod "7cd4cf93-a476-49c4-8f52-5f0b823d530a" (UID: "7cd4cf93-a476-49c4-8f52-5f0b823d530a"). InnerVolumeSpecName "kube-api-access-cn8pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:49:12.885060 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.885000    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7cd4cf93-a476-49c4-8f52-5f0b823d530a" (UID: "7cd4cf93-a476-49c4-8f52-5f0b823d530a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:49:12.927650 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.927600    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-dshm\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:49:12.927650 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.927647    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn8pg\" (UniqueName: \"kubernetes.io/projected/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kube-api-access-cn8pg\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:49:12.927917 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.927663    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cd4cf93-a476-49c4-8f52-5f0b823d530a-kserve-provision-location\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:49:12.927917 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:12.927676    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7cd4cf93-a476-49c4-8f52-5f0b823d530a-tls-certs\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:49:13.450770 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.450707    2566 generic.go:358] "Generic (PLEG): container finished" podID="7cd4cf93-a476-49c4-8f52-5f0b823d530a" containerID="b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c" exitCode=0
May 20 15:49:13.450978 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.450792    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" event={"ID":"7cd4cf93-a476-49c4-8f52-5f0b823d530a","Type":"ContainerDied","Data":"b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c"}
May 20 15:49:13.450978 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.450841    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f" event={"ID":"7cd4cf93-a476-49c4-8f52-5f0b823d530a","Type":"ContainerDied","Data":"47e40de1eaa8720ef3695d5be4130cf0bcba5c965fc4b0deff1392719ad7e700"}
May 20 15:49:13.450978 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.450806    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"
May 20 15:49:13.450978 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.450855    2566 scope.go:117] "RemoveContainer" containerID="b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c"
May 20 15:49:13.458880 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.458864    2566 scope.go:117] "RemoveContainer" containerID="e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713"
May 20 15:49:13.486669 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.486642    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"]
May 20 15:49:13.498553 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.498529    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-5d5597645b-99r2f"]
May 20 15:49:13.524140 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.524122    2566 scope.go:117] "RemoveContainer" containerID="b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c"
May 20 15:49:13.524445 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:49:13.524423    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c\": container with ID starting with b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c not found: ID does not exist" containerID="b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c"
May 20 15:49:13.524515 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.524452    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c"} err="failed to get container status \"b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c\": rpc error: code = NotFound desc = could not find container \"b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c\": container with ID starting with b340221ac9ecd1b457f11e6010a1395b980da70c5638f725b11eb4f66352eb2c not found: ID does not exist"
May 20 15:49:13.524515 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.524472    2566 scope.go:117] "RemoveContainer" containerID="e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713"
May 20 15:49:13.524719 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:49:13.524702    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713\": container with ID starting with e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713 not found: ID does not exist" containerID="e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713"
May 20 15:49:13.524783 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:13.524726    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713"} err="failed to get container status \"e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713\": rpc error: code = NotFound desc = could not find container \"e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713\": container with ID starting with e034d612df96aea7cd11e7bc4b1489110abd6b0cb1b0b8e1969af01bb17c6713 not found: ID does not exist"
May 20 15:49:14.932268 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:14.932234    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cd4cf93-a476-49c4-8f52-5f0b823d530a" path="/var/lib/kubelet/pods/7cd4cf93-a476-49c4-8f52-5f0b823d530a/volumes"
May 20 15:49:24.242723 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.242686    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"]
May 20 15:49:24.243198 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.243020    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd4cf93-a476-49c4-8f52-5f0b823d530a" containerName="storage-initializer"
May 20 15:49:24.243198 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.243031    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd4cf93-a476-49c4-8f52-5f0b823d530a" containerName="storage-initializer"
May 20 15:49:24.243198 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.243041    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd4cf93-a476-49c4-8f52-5f0b823d530a" containerName="main"
May 20 15:49:24.243198 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.243046    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd4cf93-a476-49c4-8f52-5f0b823d530a" containerName="main"
May 20 15:49:24.243198 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.243109    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="7cd4cf93-a476-49c4-8f52-5f0b823d530a" containerName="main"
May 20 15:49:24.245065 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.245050    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.249693 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.249671    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sxlvb\""
May 20 15:49:24.249855 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.249672    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
May 20 15:49:24.249855 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.249675    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
May 20 15:49:24.249855 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.249712    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
May 20 15:49:24.258480 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.258459    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"]
May 20 15:49:24.333417 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.333377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.333586 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.333443    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-home\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.333586 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.333461    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-dshm\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.333586 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.333508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.333586 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.333526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw6rl\" (UniqueName: \"kubernetes.io/projected/e52c1f4f-f306-4759-963b-29983b22236d-kube-api-access-xw6rl\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.333836 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.333670    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-model-cache\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.333836 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.333707    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e52c1f4f-f306-4759-963b-29983b22236d-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.434892 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.434860    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-model-cache\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435057 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.434896    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e52c1f4f-f306-4759-963b-29983b22236d-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435057 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.434922    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435057 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.434970    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-home\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435057 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.434992    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-dshm\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435057 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.435042    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435295 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.435066    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw6rl\" (UniqueName: \"kubernetes.io/projected/e52c1f4f-f306-4759-963b-29983b22236d-kube-api-access-xw6rl\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435345 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.435294    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-model-cache\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435345 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.435313    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435493 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.435424    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.435493 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.435428    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-home\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.437413 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.437394    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-dshm\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.437527 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.437512    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e52c1f4f-f306-4759-963b-29983b22236d-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.442999 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.442974    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw6rl\" (UniqueName: \"kubernetes.io/projected/e52c1f4f-f306-4759-963b-29983b22236d-kube-api-access-xw6rl\") pod \"scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.554769 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.554715    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:49:24.699387 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:24.699363    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"]
May 20 15:49:24.701850 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:49:24.701819    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode52c1f4f_f306_4759_963b_29983b22236d.slice/crio-6efb0e142d56b5d66f2195135ce37810c47c6456b54ea2845c5f5e04778b5282 WatchSource:0}: Error finding container 6efb0e142d56b5d66f2195135ce37810c47c6456b54ea2845c5f5e04778b5282: Status 404 returned error can't find the container with id 6efb0e142d56b5d66f2195135ce37810c47c6456b54ea2845c5f5e04778b5282
May 20 15:49:25.500593 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:25.500556    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd" event={"ID":"e52c1f4f-f306-4759-963b-29983b22236d","Type":"ContainerStarted","Data":"050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f"}
May 20 15:49:25.500593 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:49:25.500591    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd" event={"ID":"e52c1f4f-f306-4759-963b-29983b22236d","Type":"ContainerStarted","Data":"6efb0e142d56b5d66f2195135ce37810c47c6456b54ea2845c5f5e04778b5282"}
May 20 15:50:53.795657 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:50:53.795617    2566 generic.go:358] "Generic (PLEG): container finished" podID="e52c1f4f-f306-4759-963b-29983b22236d" containerID="050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f" exitCode=0
May 20 15:50:53.796111 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:50:53.795691    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd" event={"ID":"e52c1f4f-f306-4759-963b-29983b22236d","Type":"ContainerDied","Data":"050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f"}
May 20 15:50:54.800579 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:50:54.800542    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd" event={"ID":"e52c1f4f-f306-4759-963b-29983b22236d","Type":"ContainerStarted","Data":"c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c"}
May 20 15:50:54.822223 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:50:54.822168    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd" podStartSLOduration=90.822153582 podStartE2EDuration="1m30.822153582s" podCreationTimestamp="2026-05-20 15:49:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:50:54.819907987 +0000 UTC m=+1164.503777339" watchObservedRunningTime="2026-05-20 15:50:54.822153582 +0000 UTC m=+1164.506022920"
May 20 15:51:04.555629 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:04.555589    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:51:04.555629 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:04.555627    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:51:04.568347 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:04.568325    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:51:04.847253 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:04.847175    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:51:06.544290 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:06.544257    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"]
May 20 15:51:06.843023 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:06.842902    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd" podUID="e52c1f4f-f306-4759-963b-29983b22236d" containerName="main" containerID="cri-o://c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c" gracePeriod=30
May 20 15:51:07.108538 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.108513    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:51:07.117758 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.117724    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e52c1f4f-f306-4759-963b-29983b22236d-tls-certs\") pod \"e52c1f4f-f306-4759-963b-29983b22236d\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") "
May 20 15:51:07.117893 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.117773    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-home\") pod \"e52c1f4f-f306-4759-963b-29983b22236d\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") "
May 20 15:51:07.117893 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.117824    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw6rl\" (UniqueName: \"kubernetes.io/projected/e52c1f4f-f306-4759-963b-29983b22236d-kube-api-access-xw6rl\") pod \"e52c1f4f-f306-4759-963b-29983b22236d\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") "
May 20 15:51:07.117893 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.117843    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-dshm\") pod \"e52c1f4f-f306-4759-963b-29983b22236d\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") "
May 20 15:51:07.117893 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.117869    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-kserve-provision-location\") pod \"e52c1f4f-f306-4759-963b-29983b22236d\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") "
May 20 15:51:07.118096 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.117916    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-tmp-dir\") pod \"e52c1f4f-f306-4759-963b-29983b22236d\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") "
May 20 15:51:07.118096 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.117946    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-model-cache\") pod \"e52c1f4f-f306-4759-963b-29983b22236d\" (UID: \"e52c1f4f-f306-4759-963b-29983b22236d\") "
May 20 15:51:07.118197 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.118146    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-home" (OuterVolumeSpecName: "home") pod "e52c1f4f-f306-4759-963b-29983b22236d" (UID: "e52c1f4f-f306-4759-963b-29983b22236d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:51:07.118252 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.118210    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e52c1f4f-f306-4759-963b-29983b22236d" (UID: "e52c1f4f-f306-4759-963b-29983b22236d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:51:07.118337 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.118314    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-model-cache" (OuterVolumeSpecName: "model-cache") pod "e52c1f4f-f306-4759-963b-29983b22236d" (UID: "e52c1f4f-f306-4759-963b-29983b22236d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:51:07.120045 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.120025    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e52c1f4f-f306-4759-963b-29983b22236d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e52c1f4f-f306-4759-963b-29983b22236d" (UID: "e52c1f4f-f306-4759-963b-29983b22236d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:51:07.120181 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.120160    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-dshm" (OuterVolumeSpecName: "dshm") pod "e52c1f4f-f306-4759-963b-29983b22236d" (UID: "e52c1f4f-f306-4759-963b-29983b22236d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:51:07.120241 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.120166    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e52c1f4f-f306-4759-963b-29983b22236d-kube-api-access-xw6rl" (OuterVolumeSpecName: "kube-api-access-xw6rl") pod "e52c1f4f-f306-4759-963b-29983b22236d" (UID: "e52c1f4f-f306-4759-963b-29983b22236d"). InnerVolumeSpecName "kube-api-access-xw6rl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:51:07.180422 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.180381    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e52c1f4f-f306-4759-963b-29983b22236d" (UID: "e52c1f4f-f306-4759-963b-29983b22236d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:51:07.219109 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.219078    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw6rl\" (UniqueName: \"kubernetes.io/projected/e52c1f4f-f306-4759-963b-29983b22236d-kube-api-access-xw6rl\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:51:07.219109 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.219103    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-dshm\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:51:07.219109 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.219113    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-kserve-provision-location\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:51:07.219327 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.219124    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-tmp-dir\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:51:07.219327 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.219133    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-model-cache\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:51:07.219327 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.219142    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e52c1f4f-f306-4759-963b-29983b22236d-tls-certs\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:51:07.219327 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.219149    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e52c1f4f-f306-4759-963b-29983b22236d-home\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:51:07.852830 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.852797    2566 generic.go:358] "Generic (PLEG): container finished" podID="e52c1f4f-f306-4759-963b-29983b22236d" containerID="c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c" exitCode=0
May 20 15:51:07.853267 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.852878    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd" event={"ID":"e52c1f4f-f306-4759-963b-29983b22236d","Type":"ContainerDied","Data":"c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c"}
May 20 15:51:07.853267 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.852922    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd" event={"ID":"e52c1f4f-f306-4759-963b-29983b22236d","Type":"ContainerDied","Data":"6efb0e142d56b5d66f2195135ce37810c47c6456b54ea2845c5f5e04778b5282"}
May 20 15:51:07.853267 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.852941    2566 scope.go:117] "RemoveContainer" containerID="c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c"
May 20 15:51:07.853267 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.852893    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"
May 20 15:51:07.861349 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.861335    2566 scope.go:117] "RemoveContainer" containerID="050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f"
May 20 15:51:07.874962 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.874940    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"]
May 20 15:51:07.879324 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:07.879306    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-644c4fb456-fjfpd"]
May 20 15:51:08.902785 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:08.902521    2566 scope.go:117] "RemoveContainer" containerID="c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c"
May 20 15:51:08.903050 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:51:08.902832    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c\": container with ID starting with c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c not found: ID does not exist" containerID="c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c"
May 20 15:51:08.903050 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:08.902860    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c"} err="failed to get container status \"c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c\": rpc error: code = NotFound desc = could not find container \"c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c\": container with ID starting with c1a9d924602bedc8e6a9dac94ff208b8efbde19ff10514d596e495d55b19ef1c not found: ID does not exist"
May 20 15:51:08.903050 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:08.902878    2566 scope.go:117] "RemoveContainer" containerID="050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f"
May 20 15:51:08.903187 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:51:08.903139    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f\": container with ID starting with 050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f not found: ID does not exist" containerID="050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f"
May 20 15:51:08.903187 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:08.903160    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f"} err="failed to get container status \"050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f\": rpc error: code = NotFound desc = could not find container \"050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f\": container with ID starting with 050003dc22b5f8ecdfc9344b7c77b1d068a6442585c9102c1bce5f90b5d2b36f not found: ID does not exist"
May 20 15:51:08.932858 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:08.932832    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e52c1f4f-f306-4759-963b-29983b22236d" path="/var/lib/kubelet/pods/e52c1f4f-f306-4759-963b-29983b22236d/volumes"
May 20 15:51:17.923738 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.923702    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"]
May 20 15:51:17.924345 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.924094    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e52c1f4f-f306-4759-963b-29983b22236d" containerName="storage-initializer"
May 20 15:51:17.924345 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.924107    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e52c1f4f-f306-4759-963b-29983b22236d" containerName="storage-initializer"
May 20 15:51:17.924345 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.924142    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e52c1f4f-f306-4759-963b-29983b22236d" containerName="main"
May 20 15:51:17.924345 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.924148    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e52c1f4f-f306-4759-963b-29983b22236d" containerName="main"
May 20 15:51:17.924345 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.924203    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="e52c1f4f-f306-4759-963b-29983b22236d" containerName="main"
May 20 15:51:17.926250 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.926231    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:17.929362 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.929338    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
May 20 15:51:17.929476 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.929339    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
May 20 15:51:17.929476 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.929342    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-kserve-self-signed-certs\""
May 20 15:51:17.930565 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.930543    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sxlvb\""
May 20 15:51:17.942181 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:17.942155    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"]
May 20 15:51:18.006765 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.006727    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-model-cache\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.006945 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.006776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.006945 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.006843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/777f5270-4465-4430-9c6b-0c232b6e0f75-tls-certs\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.006945 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.006873    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-dshm\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.006945 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.006904    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-tmp-dir\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.007081 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.007000    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tws8\" (UniqueName: \"kubernetes.io/projected/777f5270-4465-4430-9c6b-0c232b6e0f75-kube-api-access-5tws8\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.007125 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.007104    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-home\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.108475 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.108444    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5tws8\" (UniqueName: \"kubernetes.io/projected/777f5270-4465-4430-9c6b-0c232b6e0f75-kube-api-access-5tws8\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.108654 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.108493    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-home\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.108654 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.108638    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-model-cache\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.108802 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.108680    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.108802 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.108720    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/777f5270-4465-4430-9c6b-0c232b6e0f75-tls-certs\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.108802 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.108777    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-dshm\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.108949 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.108810    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-tmp-dir\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.108949 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.108878    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-home\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.109111 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.108955    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-model-cache\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.109171 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.109159    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-tmp-dir\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.109299 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.109276    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.111267 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.111242    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-dshm\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.111408 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.111390    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/777f5270-4465-4430-9c6b-0c232b6e0f75-tls-certs\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.119914 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.119888    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tws8\" (UniqueName: \"kubernetes.io/projected/777f5270-4465-4430-9c6b-0c232b6e0f75-kube-api-access-5tws8\") pod \"precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.238039 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.237949    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:18.380917 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.380889    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"]
May 20 15:51:18.382495 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:51:18.382465    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777f5270_4465_4430_9c6b_0c232b6e0f75.slice/crio-ab9bf02d84e622191620aebafb8265dfea137278ca8277badb40c036041d7d3b WatchSource:0}: Error finding container ab9bf02d84e622191620aebafb8265dfea137278ca8277badb40c036041d7d3b: Status 404 returned error can't find the container with id ab9bf02d84e622191620aebafb8265dfea137278ca8277badb40c036041d7d3b
May 20 15:51:18.896018 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.895978    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4" event={"ID":"777f5270-4465-4430-9c6b-0c232b6e0f75","Type":"ContainerStarted","Data":"1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02"}
May 20 15:51:18.896018 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:18.896023    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4" event={"ID":"777f5270-4465-4430-9c6b-0c232b6e0f75","Type":"ContainerStarted","Data":"ab9bf02d84e622191620aebafb8265dfea137278ca8277badb40c036041d7d3b"}
May 20 15:51:22.914985 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:22.914900    2566 generic.go:358] "Generic (PLEG): container finished" podID="777f5270-4465-4430-9c6b-0c232b6e0f75" containerID="1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02" exitCode=0
May 20 15:51:22.914985 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:22.914953    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4" event={"ID":"777f5270-4465-4430-9c6b-0c232b6e0f75","Type":"ContainerDied","Data":"1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02"}
May 20 15:51:23.919841 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:23.919807    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4" event={"ID":"777f5270-4465-4430-9c6b-0c232b6e0f75","Type":"ContainerStarted","Data":"57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c"}
May 20 15:51:23.938395 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:23.938349    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4" podStartSLOduration=6.938335552 podStartE2EDuration="6.938335552s" podCreationTimestamp="2026-05-20 15:51:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 15:51:23.937325927 +0000 UTC m=+1193.621195268" watchObservedRunningTime="2026-05-20 15:51:23.938335552 +0000 UTC m=+1193.622204891"
May 20 15:51:28.238705 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:28.238670    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:28.239108 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:28.238717    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:28.251262 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:28.251239    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:28.948011 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:28.947983    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:51:30.911426 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:30.911399    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:51:30.913480 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:30.913460    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:51:30.917598 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:30.917579    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:51:30.920174 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:51:30.920157    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:52:19.308317 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.308244    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"]
May 20 15:52:19.309114 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.308553    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4" podUID="777f5270-4465-4430-9c6b-0c232b6e0f75" containerName="main" containerID="cri-o://57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c" gracePeriod=30
May 20 15:52:19.551054 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.551031    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:52:19.656548 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656464    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-model-cache\") pod \"777f5270-4465-4430-9c6b-0c232b6e0f75\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") "
May 20 15:52:19.656548 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656501    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tws8\" (UniqueName: \"kubernetes.io/projected/777f5270-4465-4430-9c6b-0c232b6e0f75-kube-api-access-5tws8\") pod \"777f5270-4465-4430-9c6b-0c232b6e0f75\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") "
May 20 15:52:19.656548 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656538    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/777f5270-4465-4430-9c6b-0c232b6e0f75-tls-certs\") pod \"777f5270-4465-4430-9c6b-0c232b6e0f75\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") "
May 20 15:52:19.656864 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656559    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-dshm\") pod \"777f5270-4465-4430-9c6b-0c232b6e0f75\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") "
May 20 15:52:19.656864 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656593    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-tmp-dir\") pod \"777f5270-4465-4430-9c6b-0c232b6e0f75\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") "
May 20 15:52:19.656864 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656621    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-kserve-provision-location\") pod \"777f5270-4465-4430-9c6b-0c232b6e0f75\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") "
May 20 15:52:19.656864 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656648    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-home\") pod \"777f5270-4465-4430-9c6b-0c232b6e0f75\" (UID: \"777f5270-4465-4430-9c6b-0c232b6e0f75\") "
May 20 15:52:19.656864 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656817    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-model-cache" (OuterVolumeSpecName: "model-cache") pod "777f5270-4465-4430-9c6b-0c232b6e0f75" (UID: "777f5270-4465-4430-9c6b-0c232b6e0f75"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:19.657087 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656945    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-model-cache\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:19.657087 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.656966    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "777f5270-4465-4430-9c6b-0c232b6e0f75" (UID: "777f5270-4465-4430-9c6b-0c232b6e0f75"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:19.657087 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.657049    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-home" (OuterVolumeSpecName: "home") pod "777f5270-4465-4430-9c6b-0c232b6e0f75" (UID: "777f5270-4465-4430-9c6b-0c232b6e0f75"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:19.658915 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.658862    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/777f5270-4465-4430-9c6b-0c232b6e0f75-kube-api-access-5tws8" (OuterVolumeSpecName: "kube-api-access-5tws8") pod "777f5270-4465-4430-9c6b-0c232b6e0f75" (UID: "777f5270-4465-4430-9c6b-0c232b6e0f75"). InnerVolumeSpecName "kube-api-access-5tws8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:52:19.659272 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.659249    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-dshm" (OuterVolumeSpecName: "dshm") pod "777f5270-4465-4430-9c6b-0c232b6e0f75" (UID: "777f5270-4465-4430-9c6b-0c232b6e0f75"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:19.659272 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.659258    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777f5270-4465-4430-9c6b-0c232b6e0f75-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "777f5270-4465-4430-9c6b-0c232b6e0f75" (UID: "777f5270-4465-4430-9c6b-0c232b6e0f75"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:52:19.714256 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.714209    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "777f5270-4465-4430-9c6b-0c232b6e0f75" (UID: "777f5270-4465-4430-9c6b-0c232b6e0f75"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:19.758147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.758112    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/777f5270-4465-4430-9c6b-0c232b6e0f75-tls-certs\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:19.758147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.758142    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-dshm\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:19.758147 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.758151    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-tmp-dir\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:19.758363 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.758160    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-kserve-provision-location\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:19.758363 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.758175    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/777f5270-4465-4430-9c6b-0c232b6e0f75-home\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:19.758363 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:19.758188    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tws8\" (UniqueName: \"kubernetes.io/projected/777f5270-4465-4430-9c6b-0c232b6e0f75-kube-api-access-5tws8\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:20.106517 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.106477    2566 generic.go:358] "Generic (PLEG): container finished" podID="777f5270-4465-4430-9c6b-0c232b6e0f75" containerID="57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c" exitCode=0
May 20 15:52:20.106721 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.106526    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4" event={"ID":"777f5270-4465-4430-9c6b-0c232b6e0f75","Type":"ContainerDied","Data":"57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c"}
May 20 15:52:20.106721 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.106574    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4" event={"ID":"777f5270-4465-4430-9c6b-0c232b6e0f75","Type":"ContainerDied","Data":"ab9bf02d84e622191620aebafb8265dfea137278ca8277badb40c036041d7d3b"}
May 20 15:52:20.106721 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.106591    2566 scope.go:117] "RemoveContainer" containerID="57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c"
May 20 15:52:20.106721 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.106629    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"
May 20 15:52:20.116077 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.116050    2566 scope.go:117] "RemoveContainer" containerID="1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02"
May 20 15:52:20.129329 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.129296    2566 scope.go:117] "RemoveContainer" containerID="57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c"
May 20 15:52:20.129611 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:52:20.129587    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c\": container with ID starting with 57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c not found: ID does not exist" containerID="57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c"
May 20 15:52:20.129706 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.129617    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c"} err="failed to get container status \"57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c\": rpc error: code = NotFound desc = could not find container \"57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c\": container with ID starting with 57c81ae6b3d3c895a2bfede1f5eb6a6cb1fcf2e1c13b99006484bdc522def59c not found: ID does not exist"
May 20 15:52:20.129706 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.129637    2566 scope.go:117] "RemoveContainer" containerID="1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02"
May 20 15:52:20.129937 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:52:20.129917    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02\": container with ID starting with 1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02 not found: ID does not exist" containerID="1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02"
May 20 15:52:20.129986 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.129943    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02"} err="failed to get container status \"1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02\": rpc error: code = NotFound desc = could not find container \"1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02\": container with ID starting with 1e41c28a5af1e56d26873248eaceab17b17d809269bd737bb42d6bf8fddc3a02 not found: ID does not exist"
May 20 15:52:20.130360 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.130329    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"]
May 20 15:52:20.135455 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.135436    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-7fbf589c6b-fw2n4"]
May 20 15:52:20.936896 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:20.936865    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="777f5270-4465-4430-9c6b-0c232b6e0f75" path="/var/lib/kubelet/pods/777f5270-4465-4430-9c6b-0c232b6e0f75/volumes"
May 20 15:52:22.710522 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.710413    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"]
May 20 15:52:22.711009 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.710988    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777f5270-4465-4430-9c6b-0c232b6e0f75" containerName="storage-initializer"
May 20 15:52:22.711061 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.711013    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="777f5270-4465-4430-9c6b-0c232b6e0f75" containerName="storage-initializer"
May 20 15:52:22.711061 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.711057    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777f5270-4465-4430-9c6b-0c232b6e0f75" containerName="main"
May 20 15:52:22.711143 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.711066    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="777f5270-4465-4430-9c6b-0c232b6e0f75" containerName="main"
May 20 15:52:22.711143 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.711139    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="777f5270-4465-4430-9c6b-0c232b6e0f75" containerName="main"
May 20 15:52:22.714473 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.714457    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.717064 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.717037    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
May 20 15:52:22.717205 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.717037    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
May 20 15:52:22.717205 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.717043    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sxlvb\""
May 20 15:52:22.717328 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.717173    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"conv-test-v1a2-to-v1a1-kserve-self-signed-certs\""
May 20 15:52:22.725222 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.725202    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"]
May 20 15:52:22.783707 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.783674    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-kserve-provision-location\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.783901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.783716    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-home\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.783901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.783762    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-model-cache\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.783901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.783820    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-tmp-dir\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.783901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.783836    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqgmw\" (UniqueName: \"kubernetes.io/projected/596e9a1b-c620-4a37-aac4-ca62176259ee-kube-api-access-wqgmw\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.783901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.783856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/596e9a1b-c620-4a37-aac4-ca62176259ee-tls-certs\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.783901 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.783892    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-dshm\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.885285 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.885237    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-dshm\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.885483 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.885317    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-kserve-provision-location\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.885483 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.885356    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-home\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.885483 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.885394    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-model-cache\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.885483 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.885428    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-tmp-dir\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.885483 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.885473    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wqgmw\" (UniqueName: \"kubernetes.io/projected/596e9a1b-c620-4a37-aac4-ca62176259ee-kube-api-access-wqgmw\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.885738 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.885505    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/596e9a1b-c620-4a37-aac4-ca62176259ee-tls-certs\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.885849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.885791    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-kserve-provision-location\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.885849 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.885823    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-home\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.886071 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.886048    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-tmp-dir\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.886071 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.886066    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-model-cache\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.887859 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.887837    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-dshm\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.888743 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.888724    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/596e9a1b-c620-4a37-aac4-ca62176259ee-tls-certs\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:22.893689 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:22.893669    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqgmw\" (UniqueName: \"kubernetes.io/projected/596e9a1b-c620-4a37-aac4-ca62176259ee-kube-api-access-wqgmw\") pod \"conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") " pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:23.026303 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:23.026257    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:23.176289 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:23.176249    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"]
May 20 15:52:23.179694 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:52:23.179666    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596e9a1b_c620_4a37_aac4_ca62176259ee.slice/crio-6702e24251f5865d68b3d0398ba4267ff68d976642087de66d72c8dfdff360bb WatchSource:0}: Error finding container 6702e24251f5865d68b3d0398ba4267ff68d976642087de66d72c8dfdff360bb: Status 404 returned error can't find the container with id 6702e24251f5865d68b3d0398ba4267ff68d976642087de66d72c8dfdff360bb
May 20 15:52:24.122208 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:24.122140    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc" event={"ID":"596e9a1b-c620-4a37-aac4-ca62176259ee","Type":"ContainerStarted","Data":"b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb"}
May 20 15:52:24.122208 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:24.122182    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc" event={"ID":"596e9a1b-c620-4a37-aac4-ca62176259ee","Type":"ContainerStarted","Data":"6702e24251f5865d68b3d0398ba4267ff68d976642087de66d72c8dfdff360bb"}
May 20 15:52:34.617393 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:34.617354    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"]
May 20 15:52:34.617871 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:34.617709    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc" podUID="596e9a1b-c620-4a37-aac4-ca62176259ee" containerName="storage-initializer" containerID="cri-o://b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb" gracePeriod=30
May 20 15:52:55.368763 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.368716    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:55.471738 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.471641    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/596e9a1b-c620-4a37-aac4-ca62176259ee-tls-certs\") pod \"596e9a1b-c620-4a37-aac4-ca62176259ee\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") "
May 20 15:52:55.471738 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.471708    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-model-cache\") pod \"596e9a1b-c620-4a37-aac4-ca62176259ee\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") "
May 20 15:52:55.471738 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.471737    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqgmw\" (UniqueName: \"kubernetes.io/projected/596e9a1b-c620-4a37-aac4-ca62176259ee-kube-api-access-wqgmw\") pod \"596e9a1b-c620-4a37-aac4-ca62176259ee\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") "
May 20 15:52:55.472049 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.471788    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-dshm\") pod \"596e9a1b-c620-4a37-aac4-ca62176259ee\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") "
May 20 15:52:55.472049 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.471832    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-kserve-provision-location\") pod \"596e9a1b-c620-4a37-aac4-ca62176259ee\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") "
May 20 15:52:55.472049 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.471889    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-home\") pod \"596e9a1b-c620-4a37-aac4-ca62176259ee\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") "
May 20 15:52:55.472049 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.471911    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-tmp-dir\") pod \"596e9a1b-c620-4a37-aac4-ca62176259ee\" (UID: \"596e9a1b-c620-4a37-aac4-ca62176259ee\") "
May 20 15:52:55.472271 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.472053    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-model-cache" (OuterVolumeSpecName: "model-cache") pod "596e9a1b-c620-4a37-aac4-ca62176259ee" (UID: "596e9a1b-c620-4a37-aac4-ca62176259ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:55.472271 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.472142    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-home" (OuterVolumeSpecName: "home") pod "596e9a1b-c620-4a37-aac4-ca62176259ee" (UID: "596e9a1b-c620-4a37-aac4-ca62176259ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:55.472271 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.472158    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-model-cache\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:55.472271 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.472235    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "596e9a1b-c620-4a37-aac4-ca62176259ee" (UID: "596e9a1b-c620-4a37-aac4-ca62176259ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:55.474044 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.474020    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596e9a1b-c620-4a37-aac4-ca62176259ee-kube-api-access-wqgmw" (OuterVolumeSpecName: "kube-api-access-wqgmw") pod "596e9a1b-c620-4a37-aac4-ca62176259ee" (UID: "596e9a1b-c620-4a37-aac4-ca62176259ee"). InnerVolumeSpecName "kube-api-access-wqgmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 15:52:55.474582 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.474564    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/596e9a1b-c620-4a37-aac4-ca62176259ee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "596e9a1b-c620-4a37-aac4-ca62176259ee" (UID: "596e9a1b-c620-4a37-aac4-ca62176259ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 15:52:55.474650 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.474582    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-dshm" (OuterVolumeSpecName: "dshm") pod "596e9a1b-c620-4a37-aac4-ca62176259ee" (UID: "596e9a1b-c620-4a37-aac4-ca62176259ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:55.534520 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.534458    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "596e9a1b-c620-4a37-aac4-ca62176259ee" (UID: "596e9a1b-c620-4a37-aac4-ca62176259ee"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 15:52:55.573188 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.573157    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/596e9a1b-c620-4a37-aac4-ca62176259ee-tls-certs\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:55.573188 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.573186    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqgmw\" (UniqueName: \"kubernetes.io/projected/596e9a1b-c620-4a37-aac4-ca62176259ee-kube-api-access-wqgmw\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:55.573357 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.573197    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-dshm\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:55.573357 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.573207    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-kserve-provision-location\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:55.573357 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.573215    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-home\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:55.573357 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:55.573223    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/596e9a1b-c620-4a37-aac4-ca62176259ee-tmp-dir\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 15:52:56.227835 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.227799    2566 generic.go:358] "Generic (PLEG): container finished" podID="596e9a1b-c620-4a37-aac4-ca62176259ee" containerID="b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb" exitCode=0
May 20 15:52:56.228028 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.227856    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc" event={"ID":"596e9a1b-c620-4a37-aac4-ca62176259ee","Type":"ContainerDied","Data":"b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb"}
May 20 15:52:56.228028 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.227869    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"
May 20 15:52:56.228028 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.227883    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc" event={"ID":"596e9a1b-c620-4a37-aac4-ca62176259ee","Type":"ContainerDied","Data":"6702e24251f5865d68b3d0398ba4267ff68d976642087de66d72c8dfdff360bb"}
May 20 15:52:56.228028 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.227899    2566 scope.go:117] "RemoveContainer" containerID="b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb"
May 20 15:52:56.273869 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.273836    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"]
May 20 15:52:56.277352 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.277327    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/conv-test-v1a2-to-v1a1-kserve-5ff8cc44b8-d5tcc"]
May 20 15:52:56.297454 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.297431    2566 scope.go:117] "RemoveContainer" containerID="b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb"
May 20 15:52:56.297845 ip-10-0-138-16 kubenswrapper[2566]: E0520 15:52:56.297821    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb\": container with ID starting with b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb not found: ID does not exist" containerID="b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb"
May 20 15:52:56.297893 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.297855    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb"} err="failed to get container status \"b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb\": rpc error: code = NotFound desc = could not find container \"b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb\": container with ID starting with b02907ae5e4608a836b148b0eb25f1b9e7cd166ebff123b094ae886c27c773eb not found: ID does not exist"
May 20 15:52:56.933142 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:52:56.933108    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="596e9a1b-c620-4a37-aac4-ca62176259ee" path="/var/lib/kubelet/pods/596e9a1b-c620-4a37-aac4-ca62176259ee/volumes"
May 20 15:56:30.934857 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:56:30.934828    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:56:30.938215 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:56:30.938187    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 15:56:30.941036 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:56:30.941019    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:56:30.943980 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:56:30.943964    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 15:58:12.738407 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.738324    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"]
May 20 15:58:12.739791 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.738657    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="596e9a1b-c620-4a37-aac4-ca62176259ee" containerName="storage-initializer"
May 20 15:58:12.739791 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.738668    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="596e9a1b-c620-4a37-aac4-ca62176259ee" containerName="storage-initializer"
May 20 15:58:12.739791 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.738735    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="596e9a1b-c620-4a37-aac4-ca62176259ee" containerName="storage-initializer"
May 20 15:58:12.740425 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.740409    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 15:58:12.746838 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.746816    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-mvg2p\""
May 20 15:58:12.746951 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.746838    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
May 20 15:58:12.751469 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.751449    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"]
May 20 15:58:12.842559 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.842528    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bee5be31-7196-4722-b937-cd7f09d41dce-cert\") pod \"llmisvc-controller-manager-54d8bbfd5f-lp8fx\" (UID: \"bee5be31-7196-4722-b937-cd7f09d41dce\") " pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 15:58:12.842724 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.842573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhg9v\" (UniqueName: \"kubernetes.io/projected/bee5be31-7196-4722-b937-cd7f09d41dce-kube-api-access-xhg9v\") pod \"llmisvc-controller-manager-54d8bbfd5f-lp8fx\" (UID: \"bee5be31-7196-4722-b937-cd7f09d41dce\") " pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 15:58:12.943550 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.943518    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bee5be31-7196-4722-b937-cd7f09d41dce-cert\") pod \"llmisvc-controller-manager-54d8bbfd5f-lp8fx\" (UID: \"bee5be31-7196-4722-b937-cd7f09d41dce\") " pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 15:58:12.943717 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.943561    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xhg9v\" (UniqueName: \"kubernetes.io/projected/bee5be31-7196-4722-b937-cd7f09d41dce-kube-api-access-xhg9v\") pod \"llmisvc-controller-manager-54d8bbfd5f-lp8fx\" (UID: \"bee5be31-7196-4722-b937-cd7f09d41dce\") " pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 15:58:12.946067 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.946048    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bee5be31-7196-4722-b937-cd7f09d41dce-cert\") pod \"llmisvc-controller-manager-54d8bbfd5f-lp8fx\" (UID: \"bee5be31-7196-4722-b937-cd7f09d41dce\") " pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 15:58:12.952157 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:12.952115    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhg9v\" (UniqueName: \"kubernetes.io/projected/bee5be31-7196-4722-b937-cd7f09d41dce-kube-api-access-xhg9v\") pod \"llmisvc-controller-manager-54d8bbfd5f-lp8fx\" (UID: \"bee5be31-7196-4722-b937-cd7f09d41dce\") " pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 15:58:13.050417 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:13.050379    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 15:58:13.175916 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:13.175892    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"]
May 20 15:58:13.178442 ip-10-0-138-16 kubenswrapper[2566]: W0520 15:58:13.178400    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbee5be31_7196_4722_b937_cd7f09d41dce.slice/crio-4818065fb91c24040367fff39ef9374be78acbc134cbc5954c9622d3ff3b96bd WatchSource:0}: Error finding container 4818065fb91c24040367fff39ef9374be78acbc134cbc5954c9622d3ff3b96bd: Status 404 returned error can't find the container with id 4818065fb91c24040367fff39ef9374be78acbc134cbc5954c9622d3ff3b96bd
May 20 15:58:13.179663 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:13.179644    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 20 15:58:13.294722 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:13.294688    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx" event={"ID":"bee5be31-7196-4722-b937-cd7f09d41dce","Type":"ContainerStarted","Data":"4818065fb91c24040367fff39ef9374be78acbc134cbc5954c9622d3ff3b96bd"}
May 20 15:58:16.307995 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:16.307960    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx" event={"ID":"bee5be31-7196-4722-b937-cd7f09d41dce","Type":"ContainerStarted","Data":"819c6ca7577c6fcf4392a4f13dda2460455a4c140eab3169c333351eb904b8b2"}
May 20 15:58:16.308404 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:16.308090    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 15:58:16.324741 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:16.324700    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx" podStartSLOduration=1.7474284180000001 podStartE2EDuration="4.324687369s" podCreationTimestamp="2026-05-20 15:58:12 +0000 UTC" firstStartedPulling="2026-05-20 15:58:13.179788912 +0000 UTC m=+1602.863658230" lastFinishedPulling="2026-05-20 15:58:15.757047853 +0000 UTC m=+1605.440917181" observedRunningTime="2026-05-20 15:58:16.323245449 +0000 UTC m=+1606.007114788" watchObservedRunningTime="2026-05-20 15:58:16.324687369 +0000 UTC m=+1606.008556719"
May 20 15:58:47.314551 ip-10-0-138-16 kubenswrapper[2566]: I0520 15:58:47.314495    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-54d8bbfd5f-lp8fx"
May 20 16:01:30.959342 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:01:30.959310    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 16:01:30.963865 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:01:30.963839    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 16:01:30.965505 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:01:30.965479    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 16:01:30.969791 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:01:30.969775    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 16:06:30.981722 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:06:30.981690    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 16:06:30.987609 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:06:30.987588    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 16:06:30.988330 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:06:30.988306    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 16:06:30.993741 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:06:30.993723    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 16:11:31.004137 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:11:31.004089    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 16:11:31.010535 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:11:31.010512    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 16:11:31.011137 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:11:31.011118    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 16:11:31.016632 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:11:31.016616    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 16:12:57.998277 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:57.998241    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
May 20 16:12:58.002398 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.002376    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.004723 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.004702    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
May 20 16:12:58.004857 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.004726    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
May 20 16:12:58.005628 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.005613    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-f312f5-cb7fb8cf-dockercfg-k7dhp\""
May 20 16:12:58.005697 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.005683    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sxlvb\""
May 20 16:12:58.005825 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.005810    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs\""
May 20 16:12:58.009839 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.009817    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
May 20 16:12:58.129431 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.129399    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.129590 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.129437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.129590 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.129550    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.129681 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.129639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.129681 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.129670    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.129778 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.129686    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fdrz\" (UniqueName: \"kubernetes.io/projected/86bc49a9-781f-4eda-83aa-90c63c75ace0-kube-api-access-6fdrz\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.129818 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.129791    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86bc49a9-781f-4eda-83aa-90c63c75ace0-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.230768 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.230724    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.230953 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.230811    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.230953 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.230836    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6fdrz\" (UniqueName: \"kubernetes.io/projected/86bc49a9-781f-4eda-83aa-90c63c75ace0-kube-api-access-6fdrz\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.230953 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.230860    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86bc49a9-781f-4eda-83aa-90c63c75ace0-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.230953 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.230912    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.231164 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.231042    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.231164 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.231123    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.231268 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.231208    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.231268 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.231216    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.231337 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.231309    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.231395 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.231378    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-home\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.233226 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.233201    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-dshm\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.233415 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.233390    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86bc49a9-781f-4eda-83aa-90c63c75ace0-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.245356 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.245332    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fdrz\" (UniqueName: \"kubernetes.io/projected/86bc49a9-781f-4eda-83aa-90c63c75ace0-kube-api-access-6fdrz\") pod \"llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.314411 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.314378    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:12:58.447475 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.447450    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
May 20 16:12:58.449256 ip-10-0-138-16 kubenswrapper[2566]: W0520 16:12:58.449217    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86bc49a9_781f_4eda_83aa_90c63c75ace0.slice/crio-02a67c82310df7b1cefe7cd60eecc142e902d865731e0686b0c368168c2062b0 WatchSource:0}: Error finding container 02a67c82310df7b1cefe7cd60eecc142e902d865731e0686b0c368168c2062b0: Status 404 returned error can't find the container with id 02a67c82310df7b1cefe7cd60eecc142e902d865731e0686b0c368168c2062b0
May 20 16:12:58.451201 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:58.451184    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 20 16:12:59.283348 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:59.283313    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"86bc49a9-781f-4eda-83aa-90c63c75ace0","Type":"ContainerStarted","Data":"83d6cd838265bcc3795cfee0c77580d71ac829436cdd61749554cb938c4ddcad"}
May 20 16:12:59.283348 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:12:59.283357    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"86bc49a9-781f-4eda-83aa-90c63c75ace0","Type":"ContainerStarted","Data":"02a67c82310df7b1cefe7cd60eecc142e902d865731e0686b0c368168c2062b0"}
May 20 16:13:03.300980 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:13:03.300944    2566 generic.go:358] "Generic (PLEG): container finished" podID="86bc49a9-781f-4eda-83aa-90c63c75ace0" containerID="83d6cd838265bcc3795cfee0c77580d71ac829436cdd61749554cb938c4ddcad" exitCode=0
May 20 16:13:03.301354 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:13:03.301014    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"86bc49a9-781f-4eda-83aa-90c63c75ace0","Type":"ContainerDied","Data":"83d6cd838265bcc3795cfee0c77580d71ac829436cdd61749554cb938c4ddcad"}
May 20 16:13:49.467969 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:13:49.467871    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"86bc49a9-781f-4eda-83aa-90c63c75ace0","Type":"ContainerStarted","Data":"4289e84b743e2e92ffb9cacec159b7c93c60c7e69cb9d3463fd61b225ffa213b"}
May 20 16:13:49.484145 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:13:49.484077    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" podStartSLOduration=6.579907415 podStartE2EDuration="52.484058813s" podCreationTimestamp="2026-05-20 16:12:57 +0000 UTC" firstStartedPulling="2026-05-20 16:13:03.302207975 +0000 UTC m=+2492.986077292" lastFinishedPulling="2026-05-20 16:13:49.20635937 +0000 UTC m=+2538.890228690" observedRunningTime="2026-05-20 16:13:49.483560794 +0000 UTC m=+2539.167430136" watchObservedRunningTime="2026-05-20 16:13:49.484058813 +0000 UTC m=+2539.167928153"
May 20 16:14:21.178718 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.178680    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"]
May 20 16:14:21.363632 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.363592    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"]
May 20 16:14:21.363820 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.363765    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.371114 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.371084    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-gw-ca-root-cert\""
May 20 16:14:21.371265 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.371067    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-2-openshift-default-dockercfg-8lq86\""
May 20 16:14:21.500730 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.500637    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-envoy\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.500730 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.500696    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-workload-certs\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.500946 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.500768    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-data\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.500946 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.500818    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-podinfo\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.500946 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.500915    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-credential-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.500946 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.500941    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.501228 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.500958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-token\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.501228 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.500975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-workload-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.501228 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.501008    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2wzl\" (UniqueName: \"kubernetes.io/projected/c51d4cd7-a858-4ba2-a071-7a83532d34d6-kube-api-access-b2wzl\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.601542 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.601499    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-podinfo\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.601735 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.601579    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-credential-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.601735 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.601608    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.601735 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.601631    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-token\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.601735 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.601649    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-workload-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.601735 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.601681    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2wzl\" (UniqueName: \"kubernetes.io/projected/c51d4cd7-a858-4ba2-a071-7a83532d34d6-kube-api-access-b2wzl\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.601735 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.601724    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-envoy\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.602102 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.601796    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-workload-certs\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.602102 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.601825    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-data\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.602253 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.602224    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-credential-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.602430 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.602355    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-workload-certs\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.602779 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.602602    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.602779 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.602699    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-data\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.602779 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.602698    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-workload-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.604557 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.604529    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-podinfo\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.604805 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.604786    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-envoy\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.610440 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.610409    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c51d4cd7-a858-4ba2-a071-7a83532d34d6-istio-token\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.610668 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.610650    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2wzl\" (UniqueName: \"kubernetes.io/projected/c51d4cd7-a858-4ba2-a071-7a83532d34d6-kube-api-access-b2wzl\") pod \"router-gateway-2-openshift-default-7bb57db77b-v78pk\" (UID: \"c51d4cd7-a858-4ba2-a071-7a83532d34d6\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.681470 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.681436    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:21.826559 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:21.826524    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"]
May 20 16:14:21.828252 ip-10-0-138-16 kubenswrapper[2566]: W0520 16:14:21.828221    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51d4cd7_a858_4ba2_a071_7a83532d34d6.slice/crio-a34cd3fca6ed3a818f4285f144306a0bab9740666b6a9d039a82b1693c8fd79d WatchSource:0}: Error finding container a34cd3fca6ed3a818f4285f144306a0bab9740666b6a9d039a82b1693c8fd79d: Status 404 returned error can't find the container with id a34cd3fca6ed3a818f4285f144306a0bab9740666b6a9d039a82b1693c8fd79d
May 20 16:14:22.586483 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:22.586445    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk" event={"ID":"c51d4cd7-a858-4ba2-a071-7a83532d34d6","Type":"ContainerStarted","Data":"a34cd3fca6ed3a818f4285f144306a0bab9740666b6a9d039a82b1693c8fd79d"}
May 20 16:14:24.663389 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:24.663342    2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 20 16:14:24.663799 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:24.663466    2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 20 16:14:24.663799 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:24.663519    2566 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 20 16:14:25.601848 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:25.601795    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk" event={"ID":"c51d4cd7-a858-4ba2-a071-7a83532d34d6","Type":"ContainerStarted","Data":"9aed8c4c92483ed061dc1d44d063ef82382bbd8304368d44f4fa513b3af1c5ef"}
May 20 16:14:25.621888 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:25.621832    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk" podStartSLOduration=1.789356438 podStartE2EDuration="4.621815959s" podCreationTimestamp="2026-05-20 16:14:21 +0000 UTC" firstStartedPulling="2026-05-20 16:14:21.830641926 +0000 UTC m=+2571.514511243" lastFinishedPulling="2026-05-20 16:14:24.663101443 +0000 UTC m=+2574.346970764" observedRunningTime="2026-05-20 16:14:25.619532663 +0000 UTC m=+2575.303402000" watchObservedRunningTime="2026-05-20 16:14:25.621815959 +0000 UTC m=+2575.305685297"
May 20 16:14:25.682181 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:25.682143    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:25.683839 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:25.683800    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk" podUID="c51d4cd7-a858-4ba2-a071-7a83532d34d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
May 20 16:14:26.682472 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:26.682427    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk" podUID="c51d4cd7-a858-4ba2-a071-7a83532d34d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
May 20 16:14:27.708509 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:27.708471    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:28.616918 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:28.616882    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:28.617991 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:28.617970    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-v78pk"
May 20 16:14:52.812522 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.812428    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"]
May 20 16:14:52.816282 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.816255    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.819539 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.819514    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
May 20 16:14:52.828991 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.827890    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"]
May 20 16:14:52.891706 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.891663    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.891937 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.891723    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.891937 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.891841    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-home\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.891937 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.891901    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-tmp-dir\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.892176 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.891955    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.892176 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.891983    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6s67\" (UniqueName: \"kubernetes.io/projected/c99bff1b-2621-41d5-9463-e416ecbad6d3-kube-api-access-b6s67\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.892176 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.892039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99bff1b-2621-41d5-9463-e416ecbad6d3-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.992669 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.992630    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.992669 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.992676    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.992981 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.992700    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-home\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.992981 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.992898    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-tmp-dir\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.992981 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.992962    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.993192 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.992992    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6s67\" (UniqueName: \"kubernetes.io/projected/c99bff1b-2621-41d5-9463-e416ecbad6d3-kube-api-access-b6s67\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.993192 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.993053    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99bff1b-2621-41d5-9463-e416ecbad6d3-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.993192 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.993173    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-home\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.993373 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.993205    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.993505 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.993477    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.993581 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.993509    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-tmp-dir\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.996006 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.995976    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99bff1b-2621-41d5-9463-e416ecbad6d3-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:52.996618 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:52.996587    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:53.001395 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:53.001372    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6s67\" (UniqueName: \"kubernetes.io/projected/c99bff1b-2621-41d5-9463-e416ecbad6d3-kube-api-access-b6s67\") pod \"router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:53.134463 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:53.134373    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:14:53.276719 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:53.276686    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"]
May 20 16:14:53.277780 ip-10-0-138-16 kubenswrapper[2566]: W0520 16:14:53.277732    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99bff1b_2621_41d5_9463_e416ecbad6d3.slice/crio-069395233b338f3013b7ac914c0a6096fcdc64c5cabc358b5673e06eaf818a1d WatchSource:0}: Error finding container 069395233b338f3013b7ac914c0a6096fcdc64c5cabc358b5673e06eaf818a1d: Status 404 returned error can't find the container with id 069395233b338f3013b7ac914c0a6096fcdc64c5cabc358b5673e06eaf818a1d
May 20 16:14:53.705302 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:53.705261    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" event={"ID":"c99bff1b-2621-41d5-9463-e416ecbad6d3","Type":"ContainerStarted","Data":"f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc"}
May 20 16:14:53.705484 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:14:53.705306    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" event={"ID":"c99bff1b-2621-41d5-9463-e416ecbad6d3","Type":"ContainerStarted","Data":"069395233b338f3013b7ac914c0a6096fcdc64c5cabc358b5673e06eaf818a1d"}
May 20 16:15:03.744180 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:03.744146    2566 generic.go:358] "Generic (PLEG): container finished" podID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerID="f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc" exitCode=0
May 20 16:15:03.744739 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:03.744226    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" event={"ID":"c99bff1b-2621-41d5-9463-e416ecbad6d3","Type":"ContainerDied","Data":"f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc"}
May 20 16:15:04.749931 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:04.749894    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" event={"ID":"c99bff1b-2621-41d5-9463-e416ecbad6d3","Type":"ContainerStarted","Data":"99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d"}
May 20 16:15:04.771808 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:04.771716    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podStartSLOduration=12.771694094 podStartE2EDuration="12.771694094s" podCreationTimestamp="2026-05-20 16:14:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 16:15:04.769149325 +0000 UTC m=+2614.453018677" watchObservedRunningTime="2026-05-20 16:15:04.771694094 +0000 UTC m=+2614.455563441"
May 20 16:15:05.565635 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:05.565598    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
May 20 16:15:05.565908 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:05.565882    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" podUID="86bc49a9-781f-4eda-83aa-90c63c75ace0" containerName="main" containerID="cri-o://4289e84b743e2e92ffb9cacec159b7c93c60c7e69cb9d3463fd61b225ffa213b" gracePeriod=30
May 20 16:15:13.135399 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:13.135357    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:15:13.135989 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:13.135416    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:15:13.137203 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:13.137173    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
May 20 16:15:23.135934 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:23.135889    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
May 20 16:15:33.135598 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:33.135545    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
May 20 16:15:35.877812 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:35.877716    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1_86bc49a9-781f-4eda-83aa-90c63c75ace0/main/0.log"
May 20 16:15:35.878234 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:35.878073    2566 generic.go:358] "Generic (PLEG): container finished" podID="86bc49a9-781f-4eda-83aa-90c63c75ace0" containerID="4289e84b743e2e92ffb9cacec159b7c93c60c7e69cb9d3463fd61b225ffa213b" exitCode=137
May 20 16:15:35.878234 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:35.878169    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"86bc49a9-781f-4eda-83aa-90c63c75ace0","Type":"ContainerDied","Data":"4289e84b743e2e92ffb9cacec159b7c93c60c7e69cb9d3463fd61b225ffa213b"}
May 20 16:15:36.244233 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.244206    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1_86bc49a9-781f-4eda-83aa-90c63c75ace0/main/0.log"
May 20 16:15:36.244603 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.244587    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:15:36.334129 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.334090    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86bc49a9-781f-4eda-83aa-90c63c75ace0-tls-certs\") pod \"86bc49a9-781f-4eda-83aa-90c63c75ace0\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") "
May 20 16:15:36.334315 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.334142    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-model-cache\") pod \"86bc49a9-781f-4eda-83aa-90c63c75ace0\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") "
May 20 16:15:36.334315 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.334162    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fdrz\" (UniqueName: \"kubernetes.io/projected/86bc49a9-781f-4eda-83aa-90c63c75ace0-kube-api-access-6fdrz\") pod \"86bc49a9-781f-4eda-83aa-90c63c75ace0\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") "
May 20 16:15:36.334315 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.334232    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-dshm\") pod \"86bc49a9-781f-4eda-83aa-90c63c75ace0\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") "
May 20 16:15:36.334315 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.334282    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-tmp-dir\") pod \"86bc49a9-781f-4eda-83aa-90c63c75ace0\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") "
May 20 16:15:36.334518 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.334337    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-kserve-provision-location\") pod \"86bc49a9-781f-4eda-83aa-90c63c75ace0\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") "
May 20 16:15:36.334518 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.334376    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-home\") pod \"86bc49a9-781f-4eda-83aa-90c63c75ace0\" (UID: \"86bc49a9-781f-4eda-83aa-90c63c75ace0\") "
May 20 16:15:36.334518 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.334398    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-model-cache" (OuterVolumeSpecName: "model-cache") pod "86bc49a9-781f-4eda-83aa-90c63c75ace0" (UID: "86bc49a9-781f-4eda-83aa-90c63c75ace0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:15:36.334715 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.334694    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-model-cache\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:15:36.335246 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.335210    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-home" (OuterVolumeSpecName: "home") pod "86bc49a9-781f-4eda-83aa-90c63c75ace0" (UID: "86bc49a9-781f-4eda-83aa-90c63c75ace0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:15:36.336674 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.336646    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-dshm" (OuterVolumeSpecName: "dshm") pod "86bc49a9-781f-4eda-83aa-90c63c75ace0" (UID: "86bc49a9-781f-4eda-83aa-90c63c75ace0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:15:36.336898 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.336879    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86bc49a9-781f-4eda-83aa-90c63c75ace0-kube-api-access-6fdrz" (OuterVolumeSpecName: "kube-api-access-6fdrz") pod "86bc49a9-781f-4eda-83aa-90c63c75ace0" (UID: "86bc49a9-781f-4eda-83aa-90c63c75ace0"). InnerVolumeSpecName "kube-api-access-6fdrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 16:15:36.337128 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.337109    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86bc49a9-781f-4eda-83aa-90c63c75ace0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "86bc49a9-781f-4eda-83aa-90c63c75ace0" (UID: "86bc49a9-781f-4eda-83aa-90c63c75ace0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 16:15:36.353493 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.353445    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "86bc49a9-781f-4eda-83aa-90c63c75ace0" (UID: "86bc49a9-781f-4eda-83aa-90c63c75ace0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:15:36.394365 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.394329    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "86bc49a9-781f-4eda-83aa-90c63c75ace0" (UID: "86bc49a9-781f-4eda-83aa-90c63c75ace0"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:15:36.435999 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.435906    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86bc49a9-781f-4eda-83aa-90c63c75ace0-tls-certs\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:15:36.435999 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.435940    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fdrz\" (UniqueName: \"kubernetes.io/projected/86bc49a9-781f-4eda-83aa-90c63c75ace0-kube-api-access-6fdrz\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:15:36.435999 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.435953    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-dshm\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:15:36.435999 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.435965    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-tmp-dir\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:15:36.435999 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.435976    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-kserve-provision-location\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:15:36.435999 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.435987    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86bc49a9-781f-4eda-83aa-90c63c75ace0-home\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:15:36.882957 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.882925    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1_86bc49a9-781f-4eda-83aa-90c63c75ace0/main/0.log"
May 20 16:15:36.883380 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.883362    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"
May 20 16:15:36.883457 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.883361    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1" event={"ID":"86bc49a9-781f-4eda-83aa-90c63c75ace0","Type":"ContainerDied","Data":"02a67c82310df7b1cefe7cd60eecc142e902d865731e0686b0c368168c2062b0"}
May 20 16:15:36.883495 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.883488    2566 scope.go:117] "RemoveContainer" containerID="4289e84b743e2e92ffb9cacec159b7c93c60c7e69cb9d3463fd61b225ffa213b"
May 20 16:15:36.893269 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.893250    2566 scope.go:117] "RemoveContainer" containerID="83d6cd838265bcc3795cfee0c77580d71ac829436cdd61749554cb938c4ddcad"
May 20 16:15:36.906415 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.906393    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
May 20 16:15:36.909521 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.909501    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1"]
May 20 16:15:36.933434 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:36.933405    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86bc49a9-781f-4eda-83aa-90c63c75ace0" path="/var/lib/kubelet/pods/86bc49a9-781f-4eda-83aa-90c63c75ace0/volumes"
May 20 16:15:43.135267 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:43.135223    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
May 20 16:15:53.135696 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:15:53.135641    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
May 20 16:16:03.135929 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:03.135885    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
May 20 16:16:13.135247 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:13.135157    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
May 20 16:16:23.135042 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:23.134986    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
May 20 16:16:31.028852 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:31.028821    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 16:16:31.035974 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:31.035946    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 16:16:31.037736 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:31.037716    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 16:16:31.044671 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:31.044651    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 16:16:33.136117 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:33.136048    2566 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
May 20 16:16:43.145347 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:43.145313    2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:16:43.153238 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:43.153208    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:16:54.696126 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:54.696085    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"]
May 20 16:16:54.697159 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:16:54.697097    2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main" containerID="cri-o://99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d" gracePeriod=30
May 20 16:17:11.667648 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:11.667612    2566 ???:1] "http: TLS handshake error from 10.0.138.16:40834: EOF"
May 20 16:17:11.671279 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:11.671253    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:11.763380 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:11.763334    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:11.770073 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:11.770031    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:12.704826 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:12.704789    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:12.767369 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:12.767340    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:12.774247 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:12.774223    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:13.714040 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:13.714012    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:13.773769 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:13.773717    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:13.780085 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:13.780057    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:14.695494 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:14.695468    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:14.760809 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:14.760781    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:14.767657 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:14.767626    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:15.701428 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:15.701402    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:15.765620 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:15.765587    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:15.771696 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:15.771671    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:16.703994 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:16.703966    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:16.765040 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:16.765003    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:16.770741 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:16.770715    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:17.697972 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:17.697936    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:17.760931 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:17.760904    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:17.767075 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:17.767052    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:18.691388 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:18.691358    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:18.753528 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:18.753496    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:18.760065 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:18.760035    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:19.680057 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:19.680021    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:19.737796 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:19.737734    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:19.744179 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:19.744150    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:20.654574 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:20.654539    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:20.714613 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:20.714580    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:20.722137 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:20.722109    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:21.674833 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:21.674803    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:21.737655 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:21.737622    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:21.743832 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:21.743806    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:22.727007 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:22.726973    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:22.788879 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:22.788843    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:22.798309 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:22.798276    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:23.775679 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:23.775645    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:23.848740 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:23.848707    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:23.858088 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:23.858062    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:24.840681 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:24.840654    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-v78pk_c51d4cd7-a858-4ba2-a071-7a83532d34d6/istio-proxy/0.log"
May 20 16:17:24.903438 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:24.903412    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/main/0.log"
May 20 16:17:24.910240 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:24.910216    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9_c99bff1b-2621-41d5-9463-e416ecbad6d3/storage-initializer/0.log"
May 20 16:17:24.944236 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:24.944211    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:17:25.025710 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.025683    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-tmp-dir\") pod \"c99bff1b-2621-41d5-9463-e416ecbad6d3\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") "
May 20 16:17:25.025886 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.025722    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-home\") pod \"c99bff1b-2621-41d5-9463-e416ecbad6d3\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") "
May 20 16:17:25.025886 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.025768    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-kserve-provision-location\") pod \"c99bff1b-2621-41d5-9463-e416ecbad6d3\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") "
May 20 16:17:25.025886 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.025787    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6s67\" (UniqueName: \"kubernetes.io/projected/c99bff1b-2621-41d5-9463-e416ecbad6d3-kube-api-access-b6s67\") pod \"c99bff1b-2621-41d5-9463-e416ecbad6d3\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") "
May 20 16:17:25.025886 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.025817    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-dshm\") pod \"c99bff1b-2621-41d5-9463-e416ecbad6d3\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") "
May 20 16:17:25.025886 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.025858    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99bff1b-2621-41d5-9463-e416ecbad6d3-tls-certs\") pod \"c99bff1b-2621-41d5-9463-e416ecbad6d3\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") "
May 20 16:17:25.026163 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.025928    2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-model-cache\") pod \"c99bff1b-2621-41d5-9463-e416ecbad6d3\" (UID: \"c99bff1b-2621-41d5-9463-e416ecbad6d3\") "
May 20 16:17:25.026387 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.026364    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "c99bff1b-2621-41d5-9463-e416ecbad6d3" (UID: "c99bff1b-2621-41d5-9463-e416ecbad6d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:17:25.026539 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.026494    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-home" (OuterVolumeSpecName: "home") pod "c99bff1b-2621-41d5-9463-e416ecbad6d3" (UID: "c99bff1b-2621-41d5-9463-e416ecbad6d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:17:25.028162 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.028131    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-dshm" (OuterVolumeSpecName: "dshm") pod "c99bff1b-2621-41d5-9463-e416ecbad6d3" (UID: "c99bff1b-2621-41d5-9463-e416ecbad6d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:17:25.028494 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.028472    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c99bff1b-2621-41d5-9463-e416ecbad6d3-kube-api-access-b6s67" (OuterVolumeSpecName: "kube-api-access-b6s67") pod "c99bff1b-2621-41d5-9463-e416ecbad6d3" (UID: "c99bff1b-2621-41d5-9463-e416ecbad6d3"). InnerVolumeSpecName "kube-api-access-b6s67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 20 16:17:25.028494 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.028479    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c99bff1b-2621-41d5-9463-e416ecbad6d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c99bff1b-2621-41d5-9463-e416ecbad6d3" (UID: "c99bff1b-2621-41d5-9463-e416ecbad6d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 20 16:17:25.045363 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.045333    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c99bff1b-2621-41d5-9463-e416ecbad6d3" (UID: "c99bff1b-2621-41d5-9463-e416ecbad6d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:17:25.086795 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.086743    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c99bff1b-2621-41d5-9463-e416ecbad6d3" (UID: "c99bff1b-2621-41d5-9463-e416ecbad6d3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 20 16:17:25.127448 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.127419    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-model-cache\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:17:25.127448 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.127446    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-tmp-dir\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:17:25.127606 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.127459    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-home\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:17:25.127606 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.127471    2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-kserve-provision-location\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:17:25.127606 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.127485    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6s67\" (UniqueName: \"kubernetes.io/projected/c99bff1b-2621-41d5-9463-e416ecbad6d3-kube-api-access-b6s67\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:17:25.127606 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.127497    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c99bff1b-2621-41d5-9463-e416ecbad6d3-dshm\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:17:25.127606 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.127509    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99bff1b-2621-41d5-9463-e416ecbad6d3-tls-certs\") on node \"ip-10-0-138-16.ec2.internal\" DevicePath \"\""
May 20 16:17:25.266979 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.266944    2566 generic.go:358] "Generic (PLEG): container finished" podID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerID="99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d" exitCode=137
May 20 16:17:25.267139 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.266982    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" event={"ID":"c99bff1b-2621-41d5-9463-e416ecbad6d3","Type":"ContainerDied","Data":"99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d"}
May 20 16:17:25.267139 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.267004    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9" event={"ID":"c99bff1b-2621-41d5-9463-e416ecbad6d3","Type":"ContainerDied","Data":"069395233b338f3013b7ac914c0a6096fcdc64c5cabc358b5673e06eaf818a1d"}
May 20 16:17:25.267139 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.267018    2566 scope.go:117] "RemoveContainer" containerID="99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d"
May 20 16:17:25.267139 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.267016    2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"
May 20 16:17:25.275912 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.275893    2566 scope.go:117] "RemoveContainer" containerID="f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc"
May 20 16:17:25.288306 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.288284    2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"]
May 20 16:17:25.288686 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.288660    2566 scope.go:117] "RemoveContainer" containerID="99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d"
May 20 16:17:25.288942 ip-10-0-138-16 kubenswrapper[2566]: E0520 16:17:25.288924    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d\": container with ID starting with 99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d not found: ID does not exist" containerID="99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d"
May 20 16:17:25.289024 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.288950    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d"} err="failed to get container status \"99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d\": rpc error: code = NotFound desc = could not find container \"99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d\": container with ID starting with 99a6c84f829104e88e3647b023b9b898908899675f90c02aab2263bc3f0b7c2d not found: ID does not exist"
May 20 16:17:25.289024 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.288969    2566 scope.go:117] "RemoveContainer" containerID="f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc"
May 20 16:17:25.289340 ip-10-0-138-16 kubenswrapper[2566]: E0520 16:17:25.289317    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc\": container with ID starting with f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc not found: ID does not exist" containerID="f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc"
May 20 16:17:25.289430 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.289346    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc"} err="failed to get container status \"f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc\": rpc error: code = NotFound desc = could not find container \"f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc\": container with ID starting with f1e87edd514b5b0dce320db010161ade01af57d7db386fbbc20d51d0e108ecfc not found: ID does not exist"
May 20 16:17:25.291727 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.291707    2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-565b6bc5f7-scgn9"]
May 20 16:17:25.955985 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:25.955948    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cc776758b-h8gbp_36a5a1fd-37ff-4e52-9774-b0ab1c033b6d/router/0.log"
May 20 16:17:26.787380 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:26.787351    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cc776758b-h8gbp_36a5a1fd-37ff-4e52-9774-b0ab1c033b6d/router/0.log"
May 20 16:17:26.933121 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:26.933089    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" path="/var/lib/kubelet/pods/c99bff1b-2621-41d5-9463-e416ecbad6d3/volumes"
May 20 16:17:27.531443 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:27.531402    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-j4s95_a6a8c445-187a-4c3b-82f3-8d58e7d53436/authorino/0.log"
May 20 16:17:27.559873 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:27.559839    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-844548ff4c-tt77j_34e00de5-0e24-4f1c-94d8-f66277ec92ea/manager/0.log"
May 20 16:17:27.628386 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:27.628355    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57dd85bb4-nclhb_92323125-b9ec-4231-beb8-180abd22ab0f/manager/0.log"
May 20 16:17:34.882106 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:34.882072    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9lvfg_df155f89-7ce7-4d42-8e53-867e99dd42f2/global-pull-secret-syncer/0.log"
May 20 16:17:35.030795 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:35.030762    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b5l6p_f11105f6-69d8-452b-9704-a4a7922fa418/konnectivity-agent/0.log"
May 20 16:17:35.103928 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:35.103895    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-16.ec2.internal_49adea86e64f32c62411eb9c86b01363/haproxy/0.log"
May 20 16:17:39.615200 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:39.615163    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68bd676465-j4s95_a6a8c445-187a-4c3b-82f3-8d58e7d53436/authorino/0.log"
May 20 16:17:39.666268 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:39.666240    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-844548ff4c-tt77j_34e00de5-0e24-4f1c-94d8-f66277ec92ea/manager/0.log"
May 20 16:17:39.759399 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:39.759369    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57dd85bb4-nclhb_92323125-b9ec-4231-beb8-180abd22ab0f/manager/0.log"
May 20 16:17:40.950927 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:40.950898    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-c9fsp_0fafb2a2-8548-4791-962c-d20781434da7/cluster-monitoring-operator/0.log"
May 20 16:17:41.119672 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.119640    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bxw26_76b66068-e932-4815-8041-1c30a059518f/node-exporter/0.log"
May 20 16:17:41.138995 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.138972    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bxw26_76b66068-e932-4815-8041-1c30a059518f/kube-rbac-proxy/0.log"
May 20 16:17:41.156733 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.156711    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bxw26_76b66068-e932-4815-8041-1c30a059518f/init-textfile/0.log"
May 20 16:17:41.473110 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.473080    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3ed908a-4eb4-4a80-8ab1-708e23e4ea00/prometheus/0.log"
May 20 16:17:41.488232 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.488190    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3ed908a-4eb4-4a80-8ab1-708e23e4ea00/config-reloader/0.log"
May 20 16:17:41.507605 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.507582    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3ed908a-4eb4-4a80-8ab1-708e23e4ea00/thanos-sidecar/0.log"
May 20 16:17:41.530176 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.530149    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3ed908a-4eb4-4a80-8ab1-708e23e4ea00/kube-rbac-proxy-web/0.log"
May 20 16:17:41.552914 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.552856    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3ed908a-4eb4-4a80-8ab1-708e23e4ea00/kube-rbac-proxy/0.log"
May 20 16:17:41.578310 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.578281    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3ed908a-4eb4-4a80-8ab1-708e23e4ea00/kube-rbac-proxy-thanos/0.log"
May 20 16:17:41.600308 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.600281    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b3ed908a-4eb4-4a80-8ab1-708e23e4ea00/init-config-reloader/0.log"
May 20 16:17:41.687082 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:41.687053    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-8hbhf_798c02f8-ccba-4c5e-895e-be114a5c569f/prometheus-operator-admission-webhook/0.log"
May 20 16:17:43.085643 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:43.085610    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-mwcm6_2409784b-f7ea-431b-9ea5-9b1e5a2fccf4/networking-console-plugin/0.log"
May 20 16:17:43.631649 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:43.631610    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/2.log"
May 20 16:17:43.639915 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:43.639888    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-lnvrg_e8582d06-acb4-42cc-bbe9-0bfa222bb480/console-operator/3.log"
May 20 16:17:44.279734 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.279703    2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"]
May 20 16:17:44.280238 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280220    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="storage-initializer"
May 20 16:17:44.280316 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280240    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="storage-initializer"
May 20 16:17:44.280316 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280256    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86bc49a9-781f-4eda-83aa-90c63c75ace0" containerName="main"
May 20 16:17:44.280316 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280264    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86bc49a9-781f-4eda-83aa-90c63c75ace0" containerName="main"
May 20 16:17:44.280316 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280281    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main"
May 20 16:17:44.280316 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280289    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main"
May 20 16:17:44.280316 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280310    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86bc49a9-781f-4eda-83aa-90c63c75ace0" containerName="storage-initializer"
May 20 16:17:44.280610 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280319    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86bc49a9-781f-4eda-83aa-90c63c75ace0" containerName="storage-initializer"
May 20 16:17:44.280610 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280397    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="c99bff1b-2621-41d5-9463-e416ecbad6d3" containerName="main"
May 20 16:17:44.280610 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.280411    2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="86bc49a9-781f-4eda-83aa-90c63c75ace0" containerName="main"
May 20 16:17:44.284641 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.284619    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.287011 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.286988    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m228h\"/\"openshift-service-ca.crt\""
May 20 16:17:44.287905 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.287855    2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m228h\"/\"default-dockercfg-br4zm\""
May 20 16:17:44.288069 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.287925    2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m228h\"/\"kube-root-ca.crt\""
May 20 16:17:44.289647 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.289588    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"]
May 20 16:17:44.395248 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.395215    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-podres\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.395432 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.395274    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-proc\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.395432 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.395299    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s4j5\" (UniqueName: \"kubernetes.io/projected/b6a4336a-9418-4c13-acb1-d09600d47061-kube-api-access-9s4j5\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.395432 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.395332    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-sys\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.395538 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.395444    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-lib-modules\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.496723 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.496691    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-sys\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.496885 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.496766    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-lib-modules\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.496885 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.496818    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-podres\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.496885 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.496835    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-sys\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.496885 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.496849    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-proc\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.496885 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.496874    2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9s4j5\" (UniqueName: \"kubernetes.io/projected/b6a4336a-9418-4c13-acb1-d09600d47061-kube-api-access-9s4j5\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.497046 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.496920    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-proc\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.497046 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.496956    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-podres\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.497046 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.496978    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6a4336a-9418-4c13-acb1-d09600d47061-lib-modules\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.504669 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.504639    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s4j5\" (UniqueName: \"kubernetes.io/projected/b6a4336a-9418-4c13-acb1-d09600d47061-kube-api-access-9s4j5\") pod \"perf-node-gather-daemonset-mnrp9\" (UID: \"b6a4336a-9418-4c13-acb1-d09600d47061\") " pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.522226 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.522194    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-96pcz_3f890232-16f6-40ea-ae7f-65f33c7ec51f/volume-data-source-validator/0.log"
May 20 16:17:44.596381 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.596291    2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:44.734182 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:44.734156    2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"]
May 20 16:17:44.737214 ip-10-0-138-16 kubenswrapper[2566]: W0520 16:17:44.737185    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb6a4336a_9418_4c13_acb1_d09600d47061.slice/crio-2aabdbedcbf857b813945512c7dffd002f6d1dae69cd86a0c629512d9c2a4227 WatchSource:0}: Error finding container 2aabdbedcbf857b813945512c7dffd002f6d1dae69cd86a0c629512d9c2a4227: Status 404 returned error can't find the container with id 2aabdbedcbf857b813945512c7dffd002f6d1dae69cd86a0c629512d9c2a4227
May 20 16:17:45.267003 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:45.266976    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-txm4g_78900876-05c6-47ea-bf8a-9ede201d5796/dns/0.log"
May 20 16:17:45.285248 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:45.285225    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-txm4g_78900876-05c6-47ea-bf8a-9ede201d5796/kube-rbac-proxy/0.log"
May 20 16:17:45.335607 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:45.335577    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9" event={"ID":"b6a4336a-9418-4c13-acb1-d09600d47061","Type":"ContainerStarted","Data":"9041b852b82334c872d61697965c0dcf75812e1008570618e0b33c7312b2765e"}
May 20 16:17:45.335607 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:45.335611    2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9" event={"ID":"b6a4336a-9418-4c13-acb1-d09600d47061","Type":"ContainerStarted","Data":"2aabdbedcbf857b813945512c7dffd002f6d1dae69cd86a0c629512d9c2a4227"}
May 20 16:17:45.335849 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:45.335692    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:45.350435 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:45.350396    2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9" podStartSLOduration=1.350381552 podStartE2EDuration="1.350381552s" podCreationTimestamp="2026-05-20 16:17:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 16:17:45.349914895 +0000 UTC m=+2775.033784235" watchObservedRunningTime="2026-05-20 16:17:45.350381552 +0000 UTC m=+2775.034250892"
May 20 16:17:45.395164 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:45.395135    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wmdhm_5e55c923-f4ed-4bef-94ef-87bfbb26bf1b/dns-node-resolver/0.log"
May 20 16:17:45.903186 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:45.903159    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w8bpd_20eb3fce-216c-4cf3-8e9a-da9a3bc6f9cf/node-ca/0.log"
May 20 16:17:46.833452 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:46.833426    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cc776758b-h8gbp_36a5a1fd-37ff-4e52-9774-b0ab1c033b6d/router/0.log"
May 20 16:17:47.298928 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:47.298898    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pf4ql_6ae3568f-2165-4389-a819-c7fd0096d88f/serve-healthcheck-canary/0.log"
May 20 16:17:47.748729 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:47.748641    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-d5829_055ee774-d1d8-4bd2-beee-c246461e5f65/insights-operator/0.log"
May 20 16:17:47.750380 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:47.750355    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-d5829_055ee774-d1d8-4bd2-beee-c246461e5f65/insights-operator/1.log"
May 20 16:17:47.835197 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:47.835167    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-75mp8_1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79/kube-rbac-proxy/0.log"
May 20 16:17:47.855126 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:47.855092    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-75mp8_1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79/exporter/0.log"
May 20 16:17:47.874525 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:47.874493    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-75mp8_1ccbb006-8c11-4e71-8ef0-8e4f2ab5bc79/extractor/0.log"
May 20 16:17:50.434020 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:50.433989    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-cf55c5997-mrlb7_29e5343b-74c2-4df6-ad9c-a0509521d761/manager/0.log"
May 20 16:17:50.455980 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:50.455955    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-9646h_3b41e364-37af-4c7e-ad47-2ea95de7bd3f/openshift-lws-operator/0.log"
May 20 16:17:51.040262 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:51.040221    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-54d8bbfd5f-lp8fx_bee5be31-7196-4722-b937-cd7f09d41dce/manager/0.log"
May 20 16:17:51.058127 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:51.058103    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-86f7b4b499-5tt55_41b6fa64-2b81-489b-91a8-65dd860aac80/server/0.log"
May 20 16:17:51.349705 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:51.349632    2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m228h/perf-node-gather-daemonset-mnrp9"
May 20 16:17:56.138724 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:56.138691    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-v4zqc_12a104cf-a5bb-402e-8b9e-af91b6c3b1be/kube-storage-version-migrator-operator/1.log"
May 20 16:17:56.140410 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:56.140380    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-v4zqc_12a104cf-a5bb-402e-8b9e-af91b6c3b1be/kube-storage-version-migrator-operator/0.log"
May 20 16:17:57.148236 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.148191    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-56bdg_7ae40396-0221-4852-a40a-ec38478dad13/kube-multus/0.log"
May 20 16:17:57.169248 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.169217    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bhg2x_0b256b7b-d193-4847-a3c7-c0868823bbae/kube-multus-additional-cni-plugins/0.log"
May 20 16:17:57.187901 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.187872    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bhg2x_0b256b7b-d193-4847-a3c7-c0868823bbae/egress-router-binary-copy/0.log"
May 20 16:17:57.208653 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.208628    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bhg2x_0b256b7b-d193-4847-a3c7-c0868823bbae/cni-plugins/0.log"
May 20 16:17:57.226703 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.226675    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bhg2x_0b256b7b-d193-4847-a3c7-c0868823bbae/bond-cni-plugin/0.log"
May 20 16:17:57.246511 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.246495    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bhg2x_0b256b7b-d193-4847-a3c7-c0868823bbae/routeoverride-cni/0.log"
May 20 16:17:57.266927 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.266908    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bhg2x_0b256b7b-d193-4847-a3c7-c0868823bbae/whereabouts-cni-bincopy/0.log"
May 20 16:17:57.285098 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.285081    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bhg2x_0b256b7b-d193-4847-a3c7-c0868823bbae/whereabouts-cni/0.log"
May 20 16:17:57.621574 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.621545    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2grdq_838a057a-e096-49a9-a086-9202ea786b05/network-metrics-daemon/0.log"
May 20 16:17:57.637480 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:57.637459    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2grdq_838a057a-e096-49a9-a086-9202ea786b05/kube-rbac-proxy/0.log"
May 20 16:17:58.912690 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:58.912655    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-controller/0.log"
May 20 16:17:58.927487 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:58.927459    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/0.log"
May 20 16:17:58.954577 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:58.954552    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovn-acl-logging/1.log"
May 20 16:17:58.973503 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:58.973474    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/kube-rbac-proxy-node/0.log"
May 20 16:17:58.994290 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:58.994266    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/kube-rbac-proxy-ovn-metrics/0.log"
May 20 16:17:59.014810 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:59.014781    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/northd/0.log"
May 20 16:17:59.032912 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:59.032892    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/nbdb/0.log"
May 20 16:17:59.051908 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:59.051887    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/sbdb/0.log"
May 20 16:17:59.236536 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:17:59.236446    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gj2r6_0e962466-c4aa-4d88-b80a-1827a1583042/ovnkube-controller/0.log"
May 20 16:18:00.542802 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:18:00.542765    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-vp9sn_cc694183-b361-413f-8726-4469b1c63ef8/check-endpoints/0.log"
May 20 16:18:00.586687 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:18:00.586656    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-mrqh2_93ab2bf4-fc4f-4508-85f9-5b7ad38aa2e8/network-check-target-container/0.log"
May 20 16:18:01.580709 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:18:01.580681    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-lc2zw_b9f73598-e4ab-4cd8-930c-a7c6a0a6d2a6/iptables-alerter/0.log"
May 20 16:18:02.316838 ip-10-0-138-16 kubenswrapper[2566]: I0520 16:18:02.316804    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-hvk6w_e89102ba-1b0e-47d7-8bae-1f53a65984c6/tuned/0.log"