May 28 15:07:43.090818 ip-10-0-135-61 systemd[1]: Starting Kubernetes Kubelet...
May 28 15:07:43.568829 ip-10-0-135-61 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 15:07:43.568829 ip-10-0-135-61 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 15:07:43.568829 ip-10-0-135-61 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 15:07:43.568829 ip-10-0-135-61 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 15:07:43.568829 ip-10-0-135-61 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 15:07:43.569726 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.569617    2560 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 15:07:43.573104 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573089    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:07:43.573104 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573103    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573107    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573111    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573114    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573117    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573120    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573123    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573128    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573132    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573136    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573139    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573142    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573145    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573148    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573150    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573153    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573156    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573163    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573166    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573169    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:07:43.573167 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573172    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573175    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573178    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573181    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573183    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573186    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573189    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573192    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573194    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573197    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573199    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573202    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573204    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573207    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573209    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573212    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573215    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573217    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573220    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:07:43.573651 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573222    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573227    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573231    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573235    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573237    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573239    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573242    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573245    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573247    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573250    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573252    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573255    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573257    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573260    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573262    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573265    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573268    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573271    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573274    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:07:43.574122 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573277    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573279    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573282    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573284    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573287    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573289    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573292    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573295    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573297    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573300    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573303    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573307    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573309    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573312    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573314    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573317    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573319    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573322    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573324    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573327    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:07:43.574605 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573329    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573332    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573334    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573337    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573339    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573342    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573344    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573732    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573739    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573742    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573745    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573748    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573751    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573754    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573756    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573759    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573761    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573765    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573767    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573771    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:07:43.575158 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573775    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573777    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573780    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573783    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573786    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573788    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573791    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573794    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573796    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573799    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573801    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573804    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573807    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573809    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573811    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573814    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573817    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573820    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573822    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573824    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:07:43.575658 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573829    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573832    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573834    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573837    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573839    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573842    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573844    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573847    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573849    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573852    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573854    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573857    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573859    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573862    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573864    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573867    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573874    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573877    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573879    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573882    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:07:43.576184 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573885    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573887    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573889    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573892    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573894    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573897    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573899    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573902    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573904    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573907    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573909    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573912    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573916    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573919    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573921    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573925    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573928    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573931    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573934    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:07:43.576691 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573937    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573940    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573942    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573945    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573947    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573950    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573952    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573955    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573957    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573960    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573962    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573965    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573967    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.573970    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574048    2560 flags.go:64] FLAG: --address="0.0.0.0"
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574056    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574062    2560 flags.go:64] FLAG: --anonymous-auth="true"
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574066    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574084    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574087    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574091    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574095    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 15:07:43.577187 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574098    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574101    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574104    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574108    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574111    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574114    2560 flags.go:64] FLAG: --cgroup-root=""
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574117    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574120    2560 flags.go:64] FLAG: --client-ca-file=""
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574123    2560 flags.go:64] FLAG: --cloud-config=""
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574125    2560 flags.go:64] FLAG: --cloud-provider="external"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574129    2560 flags.go:64] FLAG: --cluster-dns="[]"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574133    2560 flags.go:64] FLAG: --cluster-domain=""
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574136    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574140    2560 flags.go:64] FLAG: --config-dir=""
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574143    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574146    2560 flags.go:64] FLAG: --container-log-max-files="5"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574150    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574153    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574156    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574159    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574162    2560 flags.go:64] FLAG: --contention-profiling="false"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574168    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574171    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574174    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574177    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 15:07:43.577741 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574181    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574184    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574187    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574190    2560 flags.go:64] FLAG: --enable-load-reader="false"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574193    2560 flags.go:64] FLAG: --enable-server="true"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574196    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574201    2560 flags.go:64] FLAG: --event-burst="100"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574204    2560 flags.go:64] FLAG: --event-qps="50"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574207    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574210    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574213    2560 flags.go:64] FLAG: --eviction-hard=""
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574217    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574220    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574223    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574226    2560 flags.go:64] FLAG: --eviction-soft=""
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574229    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574232    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574234    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574237    2560 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574240    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574251    2560 flags.go:64] FLAG: --fail-swap-on="true"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574254    2560 flags.go:64] FLAG: --feature-gates=""
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574258    2560 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574261    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574264    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 15:07:43.578351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574267    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574270    2560 flags.go:64] FLAG: --healthz-port="10248"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574273    2560 flags.go:64] FLAG: --help="false"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574276    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-135-61.ec2.internal"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574280    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574283    2560 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574286    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574290    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574293    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574298    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574301    2560 flags.go:64] FLAG: --image-service-endpoint=""
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574303    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574306    2560 flags.go:64] FLAG: --kube-api-burst="100"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574309    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574312    2560 flags.go:64] FLAG: --kube-api-qps="50"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574315    2560 flags.go:64] FLAG: --kube-reserved=""
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574318    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574322    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574325    2560 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574327    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574330    2560 flags.go:64] FLAG: --lock-file=""
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574333    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574336    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574339    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 15:07:43.578975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574345    2560 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574348    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574350    2560 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574353    2560 flags.go:64] FLAG: --logging-format="text"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574356    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574359    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574363    2560 flags.go:64] FLAG: --manifest-url=""
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574366    2560 flags.go:64] FLAG: --manifest-url-header=""
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574370    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574373    2560 flags.go:64] FLAG: --max-open-files="1000000"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574378    2560 flags.go:64] FLAG: --max-pods="110"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574380    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574383    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574387    2560 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574390    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574393    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574395    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574400    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574406    2560 flags.go:64] FLAG: --node-status-max-images="50"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574410    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574412    2560 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574415    2560 flags.go:64] FLAG: --pod-cidr=""
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574418    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 15:07:43.579546 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574424    2560 flags.go:64] FLAG: --pod-manifest-path=""
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574427    2560 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574430    2560 flags.go:64] FLAG: --pods-per-core="0"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574433    2560 flags.go:64] FLAG: --port="10250"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574436    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574439    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00225fe4f52d4780c"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574443    2560 flags.go:64] FLAG: --qos-reserved=""
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574446    2560 flags.go:64] FLAG: --read-only-port="10255"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574449    2560 flags.go:64] FLAG: --register-node="true"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574452    2560 flags.go:64] FLAG: --register-schedulable="true"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574454    2560 flags.go:64] FLAG: --register-with-taints=""
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574458    2560 flags.go:64] FLAG: --registry-burst="10"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574461    2560 flags.go:64] FLAG: --registry-qps="5"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574464    2560 flags.go:64] FLAG: --reserved-cpus=""
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574466    2560 flags.go:64] FLAG: --reserved-memory=""
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574470    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574473    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574476    2560 flags.go:64] FLAG: --rotate-certificates="false"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574479    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574482    2560 flags.go:64] FLAG: --runonce="false"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574484    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574487    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574490    2560 flags.go:64] FLAG: --seccomp-default="false"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574493    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574496    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574499    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 15:07:43.580155 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574503    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574506    2560 flags.go:64] FLAG: --storage-driver-password="root"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574509    2560 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574512    2560 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574515    2560 flags.go:64] FLAG: --storage-driver-user="root"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574517    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574520    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574523    2560 flags.go:64] FLAG: --system-cgroups=""
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574526    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574531    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574534    2560 flags.go:64] FLAG: --tls-cert-file=""
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574537    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574541    2560 flags.go:64] FLAG: --tls-min-version=""
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574544    2560 flags.go:64] FLAG: --tls-private-key-file=""
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574548    2560 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574564    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574567    2560 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574570    2560 flags.go:64] FLAG: --v="2"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574574    2560 flags.go:64] FLAG: --version="false"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574578    2560 flags.go:64] FLAG: --vmodule=""
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574582    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574586    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574673    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574677    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:07:43.580805 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574680    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574683    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574686    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574688    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574691    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574693    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574696    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574699    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574704    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574706    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574709    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574711    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574714    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574717    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574719    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574722    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574724    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574730    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574733    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:07:43.581421 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574737    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574739    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574742    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574744    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574747    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574749    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574752    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574754    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574757    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574759    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574762    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574764    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574767    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574769    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574772    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574774    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574777    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574780    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574782    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574785    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:07:43.581924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574788    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574792    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574795    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574797    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574800    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574802    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574807    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574810    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574813    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574815    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574819    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574822    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574824    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574827    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574829    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574832    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574834    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574837    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574839    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574842    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:07:43.582417 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574844    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574864    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574867    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574870    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574873    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574876    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574878    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574881    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574884    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574887    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574889    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574892    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574894    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574900    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574903    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574906    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574908    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574911    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574913    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574916    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:07:43.582917 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574918    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574921    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574926    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574928    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.574931    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.574935    2560 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.581306    2560 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.581322    2560 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581368    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581373    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581377    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581380    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581383    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581387    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581390    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581393    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:07:43.583392 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581396    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581398    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581401    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581420    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581424    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581428    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581431    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581434    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581436    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581439    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581442    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581445    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581448    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581450    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581453    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581455    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581458    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581460    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581463    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581466    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:07:43.583815 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581469    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581471    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581474    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581477    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581480    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581482    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581485    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581488    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581491    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581493    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581496    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581499    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581501    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581504    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581507    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581509    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581513    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581516    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581520    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581522    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:07:43.584282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581525    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581527    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581531    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581533    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581536    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581538    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581541    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581543    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581546    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581548    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581572    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581574    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581577    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581580    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581583    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581586    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581589    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581592    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581595    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581598    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:07:43.584790 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581600    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581603    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581605    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581608    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581611    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581613    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581616    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581618    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581621    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581623    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581627    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581631    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581634    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581637    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581640    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581644    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581647    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:07:43.585278 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581650    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.581655    2560 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581763    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581768    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581771    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581774    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581776    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581779    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581782    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581784    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581787    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581791    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581793    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581796    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581799    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581801    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:07:43.585703 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581804    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581806    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581809    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581811    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581813    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581816    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581818    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581821    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581824    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581826    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581829    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581832    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581834    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581837    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581840    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581842    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581845    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581848    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581850    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581853    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:07:43.586132 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581855    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581858    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581860    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581863    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581866    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581868    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581871    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581874    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581877    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581879    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581882    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581884    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581887    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581889    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581892    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581894    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581896    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581899    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581901    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:07:43.586672 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581904    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581906    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581909    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581911    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581914    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581916    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581919    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581921    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581923    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581926    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581931    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581935    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581937    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581940    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581942    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581945    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581948    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581950    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581952    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581956    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:07:43.587243 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581959    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581961    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581964    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581966    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581968    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581971    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581974    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581976    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581978    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581982    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581985    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581988    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:43.581990    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.581995    2560 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.582668    2560 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 15:07:43.587748 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.585965    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 15:07:43.588140 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.586875    2560 server.go:1019] "Starting client certificate rotation"
May 28 15:07:43.588140 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.586994    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 15:07:43.588140 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.587050    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 15:07:43.610419 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.610395    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 15:07:43.613215 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.613195    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 15:07:43.634418 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.634395    2560 log.go:25] "Validated CRI v1 runtime API"
May 28 15:07:43.639838 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.639822    2560 log.go:25] "Validated CRI v1 image API"
May 28 15:07:43.641015 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.640997    2560 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 15:07:43.643829 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.643807    2560 fs.go:135] Filesystem UUIDs: map[1e933503-0afd-453b-8e5b-6b5666fd72fa:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d9305a1a-0a12-437d-b182-614762a26b32:/dev/nvme0n1p4]
May 28 15:07:43.643907 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.643827    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 15:07:43.647740 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.647720    2560 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 15:07:43.648160 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.648062    2560 manager.go:217] Machine: {Timestamp:2026-05-28 15:07:43.647566697 +0000 UTC m=+0.432717348 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098181 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fb6928d46db30e61f8244ad2254be SystemUUID:ec2fb692-8d46-db30-e61f-8244ad2254be BootID:0afac811-a929-414a-9fef-b9722b508b0c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ea:7a:d7:fa:99 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ea:7a:d7:fa:99 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:02:da:d8:6e:98 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 15:07:43.648225 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.648161    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 15:07:43.648448 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.648431    2560 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 15:07:43.651611 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.651577    2560 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 15:07:43.651743 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.651615    2560 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-61.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 15:07:43.651793 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.651753    2560 topology_manager.go:138] "Creating topology manager with none policy"
May 28 15:07:43.651793 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.651761    2560 container_manager_linux.go:306] "Creating device plugin manager"
May 28 15:07:43.651793 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.651769    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 15:07:43.651793 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.651785    2560 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 15:07:43.653075 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.653064    2560 state_mem.go:36] "Initialized new in-memory state store"
May 28 15:07:43.653193 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.653184    2560 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 15:07:43.657525 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.657513    2560 kubelet.go:491] "Attempting to sync node with API server"
May 28 15:07:43.657576 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.657537    2560 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 15:07:43.658270 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.658259    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 15:07:43.658308 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.658273    2560 kubelet.go:397] "Adding apiserver pod source"
May 28 15:07:43.658308 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.658284    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 15:07:43.659740 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.659726    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 15:07:43.659810 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.659750    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 15:07:43.662797 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.662768    2560 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 15:07:43.664799 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.664785    2560 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 15:07:43.665993 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.665980    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 15:07:43.666031 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666002    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 15:07:43.666031 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666013    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 15:07:43.666031 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666018    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 15:07:43.666031 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666027    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 15:07:43.666031 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666033    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 15:07:43.666166 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666039    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 15:07:43.666166 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666044    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 15:07:43.666166 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666052    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 15:07:43.666166 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666058    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 15:07:43.666166 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666066    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 15:07:43.666166 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.666072    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 15:07:43.667055 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.667037    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 15:07:43.667133 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.667061    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 15:07:43.668676 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.668653    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bszfs"
May 28 15:07:43.669093 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.669063    2560 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-61.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 15:07:43.669355 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.669333    2560 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 15:07:43.671379 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.671354    2560 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 15:07:43.671461 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.671400    2560 server.go:1295] "Started kubelet"
May 28 15:07:43.671527 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.671457    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 15:07:43.671596 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.671526    2560 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 15:07:43.671643 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.671616    2560 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 15:07:43.672503 ip-10-0-135-61 systemd[1]: Started Kubernetes Kubelet.
May 28 15:07:43.673089 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.672619    2560 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 15:07:43.676670 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.676577    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bszfs"
May 28 15:07:43.677217 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.677202    2560 server.go:317] "Adding debug handlers to kubelet server"
May 28 15:07:43.679345 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.679329    2560 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-61.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 15:07:43.680166 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.679263    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-61.ec2.internal.18b3c32a2e1ffd78  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-61.ec2.internal,UID:ip-10-0-135-61.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-61.ec2.internal,},FirstTimestamp:2026-05-28 15:07:43.67136908 +0000 UTC m=+0.456519730,LastTimestamp:2026-05-28 15:07:43.67136908 +0000 UTC m=+0.456519730,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-61.ec2.internal,}"
May 28 15:07:43.681869 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.681846    2560 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 15:07:43.681869 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.681856    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 15:07:43.681990 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.681874    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 15:07:43.682041 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.681989    2560 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 15:07:43.682041 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.682002    2560 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 15:07:43.682133 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.682098    2560 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 15:07:43.682183 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.682142    2560 reconstruct.go:97] "Volume reconstruction finished"
May 28 15:07:43.682183 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.682149    2560 reconciler.go:26] "Reconciler: start to sync state"
May 28 15:07:43.682678 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.682658    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:43.686619 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.686582    2560 factory.go:55] Registering systemd factory
May 28 15:07:43.686619 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.686602    2560 factory.go:223] Registration of the systemd container factory successfully
May 28 15:07:43.686989 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.686876    2560 factory.go:153] Registering CRI-O factory
May 28 15:07:43.686989 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.686891    2560 factory.go:223] Registration of the crio container factory successfully
May 28 15:07:43.686989 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.686944    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 15:07:43.686989 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.686965    2560 factory.go:103] Registering Raw factory
May 28 15:07:43.686989 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.686979    2560 manager.go:1196] Started watching for new ooms in manager
May 28 15:07:43.687368 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.687349    2560 manager.go:319] Starting recovery of all containers
May 28 15:07:43.693871 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.693848    2560 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:07:43.696823 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.696801    2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-61.ec2.internal\" not found" node="ip-10-0-135-61.ec2.internal"
May 28 15:07:43.697057 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.697004    2560 manager.go:324] Recovery completed
May 28 15:07:43.701298 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.701277    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:07:43.703655 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.703641    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:07:43.703727 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.703669    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:07:43.703727 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.703685    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientPID"
May 28 15:07:43.704181 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.704170    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 15:07:43.704236 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.704182    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 15:07:43.704236 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.704202    2560 state_mem.go:36] "Initialized new in-memory state store"
May 28 15:07:43.706352 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.706340    2560 policy_none.go:49] "None policy: Start"
May 28 15:07:43.706401 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.706356    2560 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 15:07:43.706401 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.706366    2560 state_mem.go:35] "Initializing new in-memory state store"
May 28 15:07:43.741154 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.741135    2560 manager.go:341] "Starting Device Plugin manager"
May 28 15:07:43.752540 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.741164    2560 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 15:07:43.752540 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.741173    2560 server.go:85] "Starting device plugin registration server"
May 28 15:07:43.752540 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.741392    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 15:07:43.752540 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.741418    2560 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 15:07:43.752540 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.741479    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 15:07:43.752540 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.741546    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 15:07:43.752540 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.741571    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 15:07:43.752540 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.741974    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 15:07:43.752540 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.742016    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:43.778690 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.778666    2560 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 15:07:43.780041 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.780021    2560 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 15:07:43.780115 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.780046    2560 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 15:07:43.780115 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.780062    2560 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 15:07:43.780115 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.780070    2560 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 15:07:43.780311 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.780134    2560 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 15:07:43.783490 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.783472    2560 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:07:43.842525 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.842477    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:07:43.843257 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.843242    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:07:43.843330 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.843272    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:07:43.843330 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.843284    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientPID"
May 28 15:07:43.843330 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.843311    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-61.ec2.internal"
May 28 15:07:43.851082 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.851068    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-61.ec2.internal"
May 28 15:07:43.851167 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.851087    2560 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-61.ec2.internal\": node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:43.861833 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.861816    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:43.880611 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.880586    2560 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal"]
May 28 15:07:43.880691 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.880663    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:07:43.883518 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.883504    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:07:43.883609 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.883528    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:07:43.883609 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.883537    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientPID"
May 28 15:07:43.884715 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.884703    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:07:43.884874 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.884861    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"
May 28 15:07:43.884909 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.884889    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:07:43.885414 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.885399    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:07:43.885481 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.885413    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:07:43.885481 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.885424    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:07:43.885481 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.885434    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientPID"
May 28 15:07:43.885481 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.885434    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:07:43.885481 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.885475    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientPID"
May 28 15:07:43.886665 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.886648    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal"
May 28 15:07:43.886756 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.886672    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:07:43.887339 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.887324    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:07:43.887411 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.887344    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:07:43.887411 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.887354    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeHasSufficientPID"
May 28 15:07:43.920258 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.920240    2560 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-61.ec2.internal\" not found" node="ip-10-0-135-61.ec2.internal"
May 28 15:07:43.924505 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.924489    2560 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-61.ec2.internal\" not found" node="ip-10-0-135-61.ec2.internal"
May 28 15:07:43.962837 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:43.962814    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:43.983710 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.983694    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1715a01d738f9b219bcc1d2d2749b38f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal\" (UID: \"1715a01d738f9b219bcc1d2d2749b38f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"
May 28 15:07:43.983786 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:43.983720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1715a01d738f9b219bcc1d2d2749b38f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal\" (UID: \"1715a01d738f9b219bcc1d2d2749b38f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.062962 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.062932    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.084227 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.084201    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1715a01d738f9b219bcc1d2d2749b38f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal\" (UID: \"1715a01d738f9b219bcc1d2d2749b38f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.084227 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.084222    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1715a01d738f9b219bcc1d2d2749b38f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal\" (UID: \"1715a01d738f9b219bcc1d2d2749b38f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.084348 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.084256    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1715a01d738f9b219bcc1d2d2749b38f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal\" (UID: \"1715a01d738f9b219bcc1d2d2749b38f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.084348 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.084309    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a3d58cbef8327a74cc29c171ea36677d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-61.ec2.internal\" (UID: \"a3d58cbef8327a74cc29c171ea36677d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.084414 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.084350    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1715a01d738f9b219bcc1d2d2749b38f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal\" (UID: \"1715a01d738f9b219bcc1d2d2749b38f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.163565 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.163499    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.184862 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.184838    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a3d58cbef8327a74cc29c171ea36677d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-61.ec2.internal\" (UID: \"a3d58cbef8327a74cc29c171ea36677d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.184912 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.184888    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a3d58cbef8327a74cc29c171ea36677d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-61.ec2.internal\" (UID: \"a3d58cbef8327a74cc29c171ea36677d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.222996 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.222970    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.227175 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.227155    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal"
May 28 15:07:44.263996 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.263969    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.364471 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.364433    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.464968 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.464907    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.565383 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.565360    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.588805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.588781    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 15:07:44.589351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.588934    2560 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 15:07:44.589351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.588955    2560 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 15:07:44.666359 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.666331    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.679507 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.679474    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 15:02:43 +0000 UTC" deadline="2027-11-23 02:19:20.83161409 +0000 UTC"
May 28 15:07:44.679507 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.679504    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13043h11m36.152112758s"
May 28 15:07:44.682147 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.682130    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 15:07:44.703147 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.703007    2560 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 15:07:44.727382 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.727364    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-shrln"
May 28 15:07:44.733086 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.733066    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-shrln"
May 28 15:07:44.767021 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.766998    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.828871 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:44.828830    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d58cbef8327a74cc29c171ea36677d.slice/crio-94f90e9d61badea571f4e54680955edfc8f351b594616006b78a53413f6736e7 WatchSource:0}: Error finding container 94f90e9d61badea571f4e54680955edfc8f351b594616006b78a53413f6736e7: Status 404 returned error can't find the container with id 94f90e9d61badea571f4e54680955edfc8f351b594616006b78a53413f6736e7
May 28 15:07:44.829059 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:44.829035    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1715a01d738f9b219bcc1d2d2749b38f.slice/crio-d1a49479d5098d50637dac77a5461db6e1dbaddea0fc0dd7da5da19c730db79d WatchSource:0}: Error finding container d1a49479d5098d50637dac77a5461db6e1dbaddea0fc0dd7da5da19c730db79d: Status 404 returned error can't find the container with id d1a49479d5098d50637dac77a5461db6e1dbaddea0fc0dd7da5da19c730db79d
May 28 15:07:44.832209 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.832188    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 15:07:44.867874 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.867850    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.968336 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:44.968311    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-61.ec2.internal\" not found"
May 28 15:07:44.986477 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:44.986393    2560 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:07:45.082967 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.082943    2560 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"
May 28 15:07:45.095035 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.095011    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 15:07:45.096977 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.096964    2560 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal"
May 28 15:07:45.103471 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.103447    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 15:07:45.252470 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.252096    2560 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:07:45.461394 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.461362    2560 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:07:45.659856 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.659776    2560 apiserver.go:52] "Watching apiserver"
May 28 15:07:45.666212 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.666173    2560 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 15:07:45.666537 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.666513    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-z2r4x","kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal","openshift-cluster-node-tuning-operator/tuned-46rrp","openshift-dns/node-resolver-wgnqk","openshift-multus/multus-4bxkw","openshift-multus/multus-additional-cni-plugins-q8nt5","openshift-multus/network-metrics-daemon-8jlgg","openshift-network-diagnostics/network-check-target-rmtwg","openshift-network-operator/iptables-alerter-t9f8f","kube-system/konnectivity-agent-w6hk4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw","openshift-image-registry/node-ca-g6dcp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal"]
May 28 15:07:45.669027 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.669005    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:45.669134 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:45.669098    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:07:45.670131 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.670108    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.671238 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.671219    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.672355 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.672334    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.672459 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.672433    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.672912 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.672892    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 15:07:45.673040 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.673020    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 15:07:45.673103 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.672898    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jcqlp\""
May 28 15:07:45.673745 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.673726    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k29js\""
May 28 15:07:45.673837 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.673791    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 15:07:45.673837 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.673802    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 15:07:45.675075 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.675058    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 15:07:45.675171 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.675140    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jm57m\""
May 28 15:07:45.675388 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.675372    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 15:07:45.675468 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.675437    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 15:07:45.675602 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.675587    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 15:07:45.675602 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.675380    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 15:07:45.675725 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.675641    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 15:07:45.675725 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.675676    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j5fnz\""
May 28 15:07:45.676427 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.676409    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.676506 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.676483    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:45.676599 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:45.676576    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:07:45.679303 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.678769    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.679303 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.678832    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 15:07:45.679569 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.679528    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 15:07:45.679569 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.679545    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 15:07:45.679979 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.679936    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 15:07:45.680182 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.680053    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hl9fb\""
May 28 15:07:45.680182 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.680066    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 15:07:45.680316 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.680279    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 15:07:45.680370 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.680330    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:07:45.682068 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.681201    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.683889 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.683870    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 15:07:45.683983 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.683967    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kx5qm\""
May 28 15:07:45.684111 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.684097    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 15:07:45.684397 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.684380    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 15:07:45.684545 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.684529    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8lv5g\""
May 28 15:07:45.684647 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.684530    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 15:07:45.685107 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.685085    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 15:07:45.685200 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.685178    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 15:07:45.686408 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.686386    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 15:07:45.686602 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.686570    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kfvqx\""
May 28 15:07:45.686800 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.686782    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.688371 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.688333    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 15:07:45.691081 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.691058    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 15:07:45.691211 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.691141    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7mzdh\""
May 28 15:07:45.691901 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.691867    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 15:07:45.691976 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.691902    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 15:07:45.693652 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693633    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-os-release\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.693753 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aea7a9e-2599-4468-9bec-d293c48f84a9-cni-binary-copy\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.693753 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5mz4\" (UniqueName: \"kubernetes.io/projected/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-kube-api-access-r5mz4\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.693753 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5ecd7b4a-2ba1-4753-ba17-959add9303eb-konnectivity-ca\") pod \"konnectivity-agent-w6hk4\" (UID: \"5ecd7b4a-2ba1-4753-ba17-959add9303eb\") " pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:07:45.693924 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693769    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5lsg\" (UniqueName: \"kubernetes.io/projected/5086afac-ec94-48f4-8180-672970697414-kube-api-access-j5lsg\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:45.693924 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693809    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-cnibin\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.693924 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693845    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-tuned\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.693924 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693889    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.693924 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693916    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-node-log\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693944    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5344fe70-83db-47a2-bf7f-fe4b0659ba27-ovn-node-metrics-cert\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693970    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-system-cni-dir\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.693998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-var-lib-cni-bin\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694020    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-socket-dir-parent\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694064    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-conf-dir\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694100    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9v2k\" (UniqueName: \"kubernetes.io/projected/a9714d3f-962d-4437-a7dc-c3f1322109b3-kube-api-access-k9v2k\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694127    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5ecd7b4a-2ba1-4753-ba17-959add9303eb-agent-certs\") pod \"konnectivity-agent-w6hk4\" (UID: \"5ecd7b4a-2ba1-4753-ba17-959add9303eb\") " pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-var-lib-kubelet\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694231    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-hostroot\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694256    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.694281 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694279    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/757cacce-776f-4504-94fe-de28576cf4b4-serviceca\") pod \"node-ca-g6dcp\" (UID: \"757cacce-776f-4504-94fe-de28576cf4b4\") " pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694304    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-kubernetes\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694334    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-kubelet\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694356    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-host\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694380    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8f283641-0611-4d23-b85e-28cc5090beb1-hosts-file\") pod \"node-resolver-wgnqk\" (UID: \"8f283641-0611-4d23-b85e-28cc5090beb1\") " pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694405    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-var-lib-cni-multus\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694428    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-sys-fs\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694451    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-run-systemd\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694476    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-cni-netd\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694499    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-etc-kubernetes\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694523    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1abad2b-d3f0-474f-a60a-cfa07f525595-iptables-alerter-script\") pod \"iptables-alerter-t9f8f\" (UID: \"f1abad2b-d3f0-474f-a60a-cfa07f525595\") " pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694580    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1abad2b-d3f0-474f-a60a-cfa07f525595-host-slash\") pod \"iptables-alerter-t9f8f\" (UID: \"f1abad2b-d3f0-474f-a60a-cfa07f525595\") " pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694611    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-sysconfig\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694635    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f283641-0611-4d23-b85e-28cc5090beb1-tmp-dir\") pod \"node-resolver-wgnqk\" (UID: \"8f283641-0611-4d23-b85e-28cc5090beb1\") " pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694661    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p75kq\" (UniqueName: \"kubernetes.io/projected/8f283641-0611-4d23-b85e-28cc5090beb1-kube-api-access-p75kq\") pod \"node-resolver-wgnqk\" (UID: \"8f283641-0611-4d23-b85e-28cc5090beb1\") " pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-systemd-units\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.694787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694698    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5344fe70-83db-47a2-bf7f-fe4b0659ba27-env-overrides\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694712    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-device-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694737    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lzxj\" (UniqueName: \"kubernetes.io/projected/1e40d747-fabc-4dec-9063-277ba893e6b7-kube-api-access-9lzxj\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694764    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-os-release\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694780    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-run-ovn-kubernetes\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694793    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5344fe70-83db-47a2-bf7f-fe4b0659ba27-ovnkube-config\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694810    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-sysctl-conf\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694834    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-run\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694868    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694924    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9714d3f-962d-4437-a7dc-c3f1322109b3-tmp\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694940    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-cnibin\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694962    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.694988    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-run-netns\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695012    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-log-socket\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-run-openvswitch\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.695435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695043    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-system-cni-dir\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695066    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695110    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-slash\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695175    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-daemon-config\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-registration-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695255    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695283    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-cni-bin\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695322    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krx7p\" (UniqueName: \"kubernetes.io/projected/5344fe70-83db-47a2-bf7f-fe4b0659ba27-kube-api-access-krx7p\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695371    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-var-lib-kubelet\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695396    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-run-k8s-cni-cncf-io\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-run-netns\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695442    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-cni-dir\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/757cacce-776f-4504-94fe-de28576cf4b4-host\") pod \"node-ca-g6dcp\" (UID: \"757cacce-776f-4504-94fe-de28576cf4b4\") " pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695514    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-modprobe-d\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695543    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-systemd\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-etc-openvswitch\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.696087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-run-multus-certs\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695645    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs4pp\" (UniqueName: \"kubernetes.io/projected/1aea7a9e-2599-4468-9bec-d293c48f84a9-kube-api-access-qs4pp\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-socket-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695731    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvknj\" (UniqueName: \"kubernetes.io/projected/757cacce-776f-4504-94fe-de28576cf4b4-kube-api-access-bvknj\") pod \"node-ca-g6dcp\" (UID: \"757cacce-776f-4504-94fe-de28576cf4b4\") " pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695765    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-sys\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695799    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5344fe70-83db-47a2-bf7f-fe4b0659ba27-ovnkube-script-lib\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695825    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgrjj\" (UniqueName: \"kubernetes.io/projected/f1abad2b-d3f0-474f-a60a-cfa07f525595-kube-api-access-mgrjj\") pod \"iptables-alerter-t9f8f\" (UID: \"f1abad2b-d3f0-474f-a60a-cfa07f525595\") " pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695897    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-sysctl-d\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695922    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-lib-modules\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695945    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-run-ovn\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.695977    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.696766 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.696005    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-var-lib-openvswitch\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.734057 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.734022    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 15:02:44 +0000 UTC" deadline="2027-11-26 17:28:44.801209032 +0000 UTC"
May 28 15:07:45.734158 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.734055    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13130h20m59.067158184s"
May 28 15:07:45.779152 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.779113    2560 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:07:45.782875 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.782856    2560 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 15:07:45.785328 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.785283    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal" event={"ID":"a3d58cbef8327a74cc29c171ea36677d","Type":"ContainerStarted","Data":"94f90e9d61badea571f4e54680955edfc8f351b594616006b78a53413f6736e7"}
May 28 15:07:45.786421 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.786394    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal" event={"ID":"1715a01d738f9b219bcc1d2d2749b38f","Type":"ContainerStarted","Data":"d1a49479d5098d50637dac77a5461db6e1dbaddea0fc0dd7da5da19c730db79d"}
May 28 15:07:45.796871 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.796846    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.796978 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.796889    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9714d3f-962d-4437-a7dc-c3f1322109b3-tmp\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.796978 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.796910    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-cnibin\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.796978 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.796919    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.796978 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.796931    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.796988    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-run-netns\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797020    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-cnibin\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797014    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-log-socket\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797058    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-run-openvswitch\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797078    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-run-netns\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797084    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-system-cni-dir\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797120    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797143    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-system-cni-dir\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797147    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-slash\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797172    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-daemon-config\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797184    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-run-openvswitch\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797530 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797145    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-log-socket\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797530 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797201    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-registration-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.797530 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797226    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.797530 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797253    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-cni-bin\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797530 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797269    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 15:07:45.797530 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797231    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-slash\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797530 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797276    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-registration-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.797530 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797435    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.797530 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797500    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-cni-bin\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797275    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-krx7p\" (UniqueName: \"kubernetes.io/projected/5344fe70-83db-47a2-bf7f-fe4b0659ba27-kube-api-access-krx7p\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797591    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-var-lib-kubelet\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797624    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-run-k8s-cni-cncf-io\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797650    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-run-netns\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797696    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-cni-dir\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797711    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-var-lib-kubelet\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797732    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/757cacce-776f-4504-94fe-de28576cf4b4-host\") pod \"node-ca-g6dcp\" (UID: \"757cacce-776f-4504-94fe-de28576cf4b4\") " pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797743    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-run-netns\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797758    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-modprobe-d\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797773    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-run-k8s-cni-cncf-io\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797778    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-daemon-config\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797785    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-systemd\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797807    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-etc-openvswitch\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797823    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-cni-dir\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797828    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-run-multus-certs\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797852    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qs4pp\" (UniqueName: \"kubernetes.io/projected/1aea7a9e-2599-4468-9bec-d293c48f84a9-kube-api-access-qs4pp\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797863    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-systemd\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.797951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797875    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-socket-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797888    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-modprobe-d\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797896    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-run-multus-certs\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797903    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bvknj\" (UniqueName: \"kubernetes.io/projected/757cacce-776f-4504-94fe-de28576cf4b4-kube-api-access-bvknj\") pod \"node-ca-g6dcp\" (UID: \"757cacce-776f-4504-94fe-de28576cf4b4\") " pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797925    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-etc-openvswitch\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797927    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-sys\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797933    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797956    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5344fe70-83db-47a2-bf7f-fe4b0659ba27-ovnkube-script-lib\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797928    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/757cacce-776f-4504-94fe-de28576cf4b4-host\") pod \"node-ca-g6dcp\" (UID: \"757cacce-776f-4504-94fe-de28576cf4b4\") " pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.797978    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mgrjj\" (UniqueName: \"kubernetes.io/projected/f1abad2b-d3f0-474f-a60a-cfa07f525595-kube-api-access-mgrjj\") pod \"iptables-alerter-t9f8f\" (UID: \"f1abad2b-d3f0-474f-a60a-cfa07f525595\") " pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798014    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-socket-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798058    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-sys\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798211    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-sysctl-d\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798236    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-lib-modules\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798258    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-run-ovn\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798306    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-sysctl-d\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798297    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.798859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798344    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-var-lib-openvswitch\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798361    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-run-ovn\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798371    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-os-release\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798421    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-lib-modules\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798435    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-os-release\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798452    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5344fe70-83db-47a2-bf7f-fe4b0659ba27-ovnkube-script-lib\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798469    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aea7a9e-2599-4468-9bec-d293c48f84a9-cni-binary-copy\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798502    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r5mz4\" (UniqueName: \"kubernetes.io/projected/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-kube-api-access-r5mz4\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798497    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-var-lib-openvswitch\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798534    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5ecd7b4a-2ba1-4753-ba17-959add9303eb-konnectivity-ca\") pod \"konnectivity-agent-w6hk4\" (UID: \"5ecd7b4a-2ba1-4753-ba17-959add9303eb\") " pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798597    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j5lsg\" (UniqueName: \"kubernetes.io/projected/5086afac-ec94-48f4-8180-672970697414-kube-api-access-j5lsg\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798774    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-cnibin\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798796    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-tuned\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798812    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798819    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798843    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-node-log\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798868    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5344fe70-83db-47a2-bf7f-fe4b0659ba27-ovn-node-metrics-cert\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.799673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798877    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-cnibin\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798891    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-system-cni-dir\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798912    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-var-lib-cni-bin\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798912    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aea7a9e-2599-4468-9bec-d293c48f84a9-cni-binary-copy\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798950    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-node-log\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.798980    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799007    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-socket-dir-parent\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799035    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-conf-dir\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799067    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k9v2k\" (UniqueName: \"kubernetes.io/projected/a9714d3f-962d-4437-a7dc-c3f1322109b3-kube-api-access-k9v2k\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799090    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5ecd7b4a-2ba1-4753-ba17-959add9303eb-agent-certs\") pod \"konnectivity-agent-w6hk4\" (UID: \"5ecd7b4a-2ba1-4753-ba17-959add9303eb\") " pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799115    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-var-lib-kubelet\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799141    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-hostroot\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799166    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799189    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/757cacce-776f-4504-94fe-de28576cf4b4-serviceca\") pod \"node-ca-g6dcp\" (UID: \"757cacce-776f-4504-94fe-de28576cf4b4\") " pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799213    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-kubernetes\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799236    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-kubelet\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799263    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-host\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799288    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8f283641-0611-4d23-b85e-28cc5090beb1-hosts-file\") pod \"node-resolver-wgnqk\" (UID: \"8f283641-0611-4d23-b85e-28cc5090beb1\") " pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.800428 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799312    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-var-lib-cni-multus\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799340    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-sys-fs\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799340    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-system-cni-dir\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799362    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-var-lib-cni-bin\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799420    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-socket-dir-parent\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799365    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-run-systemd\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799434    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-run-systemd\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799163    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799459    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-cni-netd\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799491    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-cni-netd\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:45.799495    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799492    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-etc-kubernetes\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799529    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-var-lib-kubelet\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799598    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-hostroot\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799601    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-etc-kubernetes\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:45.799619    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs podName:5086afac-ec94-48f4-8180-672970697414 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:46.299538373 +0000 UTC m=+3.084689011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs") pod "network-metrics-daemon-8jlgg" (UID: "5086afac-ec94-48f4-8180-672970697414") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799648    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1abad2b-d3f0-474f-a60a-cfa07f525595-iptables-alerter-script\") pod \"iptables-alerter-t9f8f\" (UID: \"f1abad2b-d3f0-474f-a60a-cfa07f525595\") " pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.801125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799654    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799657    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-multus-conf-dir\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799681    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1abad2b-d3f0-474f-a60a-cfa07f525595-host-slash\") pod \"iptables-alerter-t9f8f\" (UID: \"f1abad2b-d3f0-474f-a60a-cfa07f525595\") " pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799701    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8f283641-0611-4d23-b85e-28cc5090beb1-hosts-file\") pod \"node-resolver-wgnqk\" (UID: \"8f283641-0611-4d23-b85e-28cc5090beb1\") " pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799708    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-sysconfig\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799732    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f283641-0611-4d23-b85e-28cc5090beb1-tmp-dir\") pod \"node-resolver-wgnqk\" (UID: \"8f283641-0611-4d23-b85e-28cc5090beb1\") " pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799755    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p75kq\" (UniqueName: \"kubernetes.io/projected/8f283641-0611-4d23-b85e-28cc5090beb1-kube-api-access-p75kq\") pod \"node-resolver-wgnqk\" (UID: \"8f283641-0611-4d23-b85e-28cc5090beb1\") " pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799779    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-systemd-units\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799803    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5344fe70-83db-47a2-bf7f-fe4b0659ba27-env-overrides\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799827    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-device-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799832    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aea7a9e-2599-4468-9bec-d293c48f84a9-host-var-lib-cni-multus\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799313    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5ecd7b4a-2ba1-4753-ba17-959add9303eb-konnectivity-ca\") pod \"konnectivity-agent-w6hk4\" (UID: \"5ecd7b4a-2ba1-4753-ba17-959add9303eb\") " pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799873    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9lzxj\" (UniqueName: \"kubernetes.io/projected/1e40d747-fabc-4dec-9063-277ba893e6b7-kube-api-access-9lzxj\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.799906    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-sys-fs\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800078    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/757cacce-776f-4504-94fe-de28576cf4b4-serviceca\") pod \"node-ca-g6dcp\" (UID: \"757cacce-776f-4504-94fe-de28576cf4b4\") " pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800110    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-systemd-units\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800147    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-kubernetes\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800164    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f283641-0611-4d23-b85e-28cc5090beb1-tmp-dir\") pod \"node-resolver-wgnqk\" (UID: \"8f283641-0611-4d23-b85e-28cc5090beb1\") " pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.801958 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800194    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-kubelet\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800252    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-host\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800277    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1abad2b-d3f0-474f-a60a-cfa07f525595-host-slash\") pod \"iptables-alerter-t9f8f\" (UID: \"f1abad2b-d3f0-474f-a60a-cfa07f525595\") " pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800303    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-sysconfig\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800322    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-os-release\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800351    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-run-ovn-kubernetes\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800356    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e40d747-fabc-4dec-9063-277ba893e6b7-device-dir\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800370    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5344fe70-83db-47a2-bf7f-fe4b0659ba27-ovnkube-config\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800395    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-sysctl-conf\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800415    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-os-release\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800414    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1abad2b-d3f0-474f-a60a-cfa07f525595-iptables-alerter-script\") pod \"iptables-alerter-t9f8f\" (UID: \"f1abad2b-d3f0-474f-a60a-cfa07f525595\") " pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800418    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-run\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800453    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-run\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800457    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5344fe70-83db-47a2-bf7f-fe4b0659ba27-env-overrides\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800458    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800488    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800528    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5344fe70-83db-47a2-bf7f-fe4b0659ba27-host-run-ovn-kubernetes\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.802849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.801481    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5344fe70-83db-47a2-bf7f-fe4b0659ba27-ovnkube-config\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.803743 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.801751    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9714d3f-962d-4437-a7dc-c3f1322109b3-tmp\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.803743 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.802964    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-tuned\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.803743 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.803080    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5344fe70-83db-47a2-bf7f-fe4b0659ba27-ovn-node-metrics-cert\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.803743 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.803211    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5ecd7b4a-2ba1-4753-ba17-959add9303eb-agent-certs\") pod \"konnectivity-agent-w6hk4\" (UID: \"5ecd7b4a-2ba1-4753-ba17-959add9303eb\") " pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:07:45.805035 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.800627    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9714d3f-962d-4437-a7dc-c3f1322109b3-etc-sysctl-conf\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.810204 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:45.809935    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:07:45.810204 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:45.809959    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:07:45.810204 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:45.809975    2560 projected.go:194] Error preparing data for projected volume kube-api-access-hf5br for pod openshift-network-diagnostics/network-check-target-rmtwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:45.810204 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:45.810043    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br podName:237da9f1-53ff-4878-9254-e0a1cc9a01db nodeName:}" failed. No retries permitted until 2026-05-28 15:07:46.310027209 +0000 UTC m=+3.095177851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hf5br" (UniqueName: "kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br") pod "network-check-target-rmtwg" (UID: "237da9f1-53ff-4878-9254-e0a1cc9a01db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:45.812985 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.812958    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krx7p\" (UniqueName: \"kubernetes.io/projected/5344fe70-83db-47a2-bf7f-fe4b0659ba27-kube-api-access-krx7p\") pod \"ovnkube-node-z2r4x\" (UID: \"5344fe70-83db-47a2-bf7f-fe4b0659ba27\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:45.815624 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.815603    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgrjj\" (UniqueName: \"kubernetes.io/projected/f1abad2b-d3f0-474f-a60a-cfa07f525595-kube-api-access-mgrjj\") pod \"iptables-alerter-t9f8f\" (UID: \"f1abad2b-d3f0-474f-a60a-cfa07f525595\") " pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:45.815838 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.815733    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5mz4\" (UniqueName: \"kubernetes.io/projected/30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc-kube-api-access-r5mz4\") pod \"multus-additional-cni-plugins-q8nt5\" (UID: \"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc\") " pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:45.816244 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.816218    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs4pp\" (UniqueName: \"kubernetes.io/projected/1aea7a9e-2599-4468-9bec-d293c48f84a9-kube-api-access-qs4pp\") pod \"multus-4bxkw\" (UID: \"1aea7a9e-2599-4468-9bec-d293c48f84a9\") " pod="openshift-multus/multus-4bxkw"
May 28 15:07:45.816320 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.816286    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p75kq\" (UniqueName: \"kubernetes.io/projected/8f283641-0611-4d23-b85e-28cc5090beb1-kube-api-access-p75kq\") pod \"node-resolver-wgnqk\" (UID: \"8f283641-0611-4d23-b85e-28cc5090beb1\") " pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:45.816634 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.816613    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9v2k\" (UniqueName: \"kubernetes.io/projected/a9714d3f-962d-4437-a7dc-c3f1322109b3-kube-api-access-k9v2k\") pod \"tuned-46rrp\" (UID: \"a9714d3f-962d-4437-a7dc-c3f1322109b3\") " pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.816727 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.816705    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lzxj\" (UniqueName: \"kubernetes.io/projected/1e40d747-fabc-4dec-9063-277ba893e6b7-kube-api-access-9lzxj\") pod \"aws-ebs-csi-driver-node-hv6hw\" (UID: \"1e40d747-fabc-4dec-9063-277ba893e6b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:45.817779 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.817654    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5lsg\" (UniqueName: \"kubernetes.io/projected/5086afac-ec94-48f4-8180-672970697414-kube-api-access-j5lsg\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:45.817779 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.817737    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvknj\" (UniqueName: \"kubernetes.io/projected/757cacce-776f-4504-94fe-de28576cf4b4-kube-api-access-bvknj\") pod \"node-ca-g6dcp\" (UID: \"757cacce-776f-4504-94fe-de28576cf4b4\") " pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:45.983668 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.983586    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-46rrp"
May 28 15:07:45.992408 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:45.992377    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wgnqk"
May 28 15:07:46.000396 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.000370    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4bxkw"
May 28 15:07:46.011128 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.011106    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q8nt5"
May 28 15:07:46.017771 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.017753    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:07:46.025362 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.025343    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t9f8f"
May 28 15:07:46.031954 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.031935    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:07:46.038515 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.038496    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw"
May 28 15:07:46.045060 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.045036    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g6dcp"
May 28 15:07:46.303918 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.303844    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:46.304072 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:46.303956    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:46.304072 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:46.304008    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs podName:5086afac-ec94-48f4-8180-672970697414 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:47.303994072 +0000 UTC m=+4.089144709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs") pod "network-metrics-daemon-8jlgg" (UID: "5086afac-ec94-48f4-8180-672970697414") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:46.404513 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.404489    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:46.404704 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:46.404685    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:07:46.404769 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:46.404713    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:07:46.404769 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:46.404729    2560 projected.go:194] Error preparing data for projected volume kube-api-access-hf5br for pod openshift-network-diagnostics/network-check-target-rmtwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:46.404874 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:46.404798    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br podName:237da9f1-53ff-4878-9254-e0a1cc9a01db nodeName:}" failed. No retries permitted until 2026-05-28 15:07:47.40477834 +0000 UTC m=+4.189928981 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf5br" (UniqueName: "kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br") pod "network-check-target-rmtwg" (UID: "237da9f1-53ff-4878-9254-e0a1cc9a01db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:46.405884 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:46.405841    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d8e3e3_d6e7_4f9d_a050_a302d06bf8bc.slice/crio-6c7cb73041b241133f1233c5370f89a8a8edce6d39c15e8164ae646fdd115f2a WatchSource:0}: Error finding container 6c7cb73041b241133f1233c5370f89a8a8edce6d39c15e8164ae646fdd115f2a: Status 404 returned error can't find the container with id 6c7cb73041b241133f1233c5370f89a8a8edce6d39c15e8164ae646fdd115f2a
May 28 15:07:46.407762 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:46.407715    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod757cacce_776f_4504_94fe_de28576cf4b4.slice/crio-395f877c107e847846fc52b8162fbc5bbb97fc521ca588ec7d0f2c60f5dfc9f0 WatchSource:0}: Error finding container 395f877c107e847846fc52b8162fbc5bbb97fc521ca588ec7d0f2c60f5dfc9f0: Status 404 returned error can't find the container with id 395f877c107e847846fc52b8162fbc5bbb97fc521ca588ec7d0f2c60f5dfc9f0
May 28 15:07:46.409257 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:46.409232    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e40d747_fabc_4dec_9063_277ba893e6b7.slice/crio-303926f7bbb937f60739135dc33d04947c45a9dd2f184211b4746a7981401e19 WatchSource:0}: Error finding container 303926f7bbb937f60739135dc33d04947c45a9dd2f184211b4746a7981401e19: Status 404 returned error can't find the container with id 303926f7bbb937f60739135dc33d04947c45a9dd2f184211b4746a7981401e19
May 28 15:07:46.412019 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:46.411995    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aea7a9e_2599_4468_9bec_d293c48f84a9.slice/crio-08ac9f35faa2934ff6c2441b588747a09a1e8747afa51aaae1ac99685a25a688 WatchSource:0}: Error finding container 08ac9f35faa2934ff6c2441b588747a09a1e8747afa51aaae1ac99685a25a688: Status 404 returned error can't find the container with id 08ac9f35faa2934ff6c2441b588747a09a1e8747afa51aaae1ac99685a25a688
May 28 15:07:46.413233 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:46.413212    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1abad2b_d3f0_474f_a60a_cfa07f525595.slice/crio-214aadd0f78817feb4fc94b6f3b0f8b7f44e4b9d8880ee94a54708b1352d11e3 WatchSource:0}: Error finding container 214aadd0f78817feb4fc94b6f3b0f8b7f44e4b9d8880ee94a54708b1352d11e3: Status 404 returned error can't find the container with id 214aadd0f78817feb4fc94b6f3b0f8b7f44e4b9d8880ee94a54708b1352d11e3
May 28 15:07:46.414282 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:46.414259    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5344fe70_83db_47a2_bf7f_fe4b0659ba27.slice/crio-c7fb5862f31121efdd35bca55c4a29d08a0e5d7b20bab6738d1209f8a32447ce WatchSource:0}: Error finding container c7fb5862f31121efdd35bca55c4a29d08a0e5d7b20bab6738d1209f8a32447ce: Status 404 returned error can't find the container with id c7fb5862f31121efdd35bca55c4a29d08a0e5d7b20bab6738d1209f8a32447ce
May 28 15:07:46.420245 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:07:46.417033    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f283641_0611_4d23_b85e_28cc5090beb1.slice/crio-25a75053c24ef7640bfbc8df06b088ddfb37177ae26071685a70ce5406b668de WatchSource:0}: Error finding container 25a75053c24ef7640bfbc8df06b088ddfb37177ae26071685a70ce5406b668de: Status 404 returned error can't find the container with id 25a75053c24ef7640bfbc8df06b088ddfb37177ae26071685a70ce5406b668de
May 28 15:07:46.734589 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.734537    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 15:02:44 +0000 UTC" deadline="2028-02-28 16:18:26.689893484 +0000 UTC"
May 28 15:07:46.734589 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.734586    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15385h10m39.955310447s"
May 28 15:07:46.790439 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.790386    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" event={"ID":"5344fe70-83db-47a2-bf7f-fe4b0659ba27","Type":"ContainerStarted","Data":"c7fb5862f31121efdd35bca55c4a29d08a0e5d7b20bab6738d1209f8a32447ce"}
May 28 15:07:46.791580 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.791506    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw" event={"ID":"1e40d747-fabc-4dec-9063-277ba893e6b7","Type":"ContainerStarted","Data":"303926f7bbb937f60739135dc33d04947c45a9dd2f184211b4746a7981401e19"}
May 28 15:07:46.794533 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.794275    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g6dcp" event={"ID":"757cacce-776f-4504-94fe-de28576cf4b4","Type":"ContainerStarted","Data":"395f877c107e847846fc52b8162fbc5bbb97fc521ca588ec7d0f2c60f5dfc9f0"}
May 28 15:07:46.801135 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.801108    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal" event={"ID":"a3d58cbef8327a74cc29c171ea36677d","Type":"ContainerStarted","Data":"b5335dec83bc946d2e06f1668b62ca07bbfbf5e09495db79b911f2c7eac4b042"}
May 28 15:07:46.803179 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.803143    2560 generic.go:358] "Generic (PLEG): container finished" podID="1715a01d738f9b219bcc1d2d2749b38f" containerID="9989d4f40ef5cdd448ff75f0b07f75e9a157701dc005d326fd7b9d4810fab940" exitCode=0
May 28 15:07:46.803399 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.803378    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal" event={"ID":"1715a01d738f9b219bcc1d2d2749b38f","Type":"ContainerDied","Data":"9989d4f40ef5cdd448ff75f0b07f75e9a157701dc005d326fd7b9d4810fab940"}
May 28 15:07:46.806249 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.806222    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t9f8f" event={"ID":"f1abad2b-d3f0-474f-a60a-cfa07f525595","Type":"ContainerStarted","Data":"214aadd0f78817feb4fc94b6f3b0f8b7f44e4b9d8880ee94a54708b1352d11e3"}
May 28 15:07:46.808348 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.808324    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-46rrp" event={"ID":"a9714d3f-962d-4437-a7dc-c3f1322109b3","Type":"ContainerStarted","Data":"de7d21cd63ea795f29d6c20a4ae59383e1038e590fb1884cc2af4a19b344599d"}
May 28 15:07:46.810326 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.810304    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4bxkw" event={"ID":"1aea7a9e-2599-4468-9bec-d293c48f84a9","Type":"ContainerStarted","Data":"08ac9f35faa2934ff6c2441b588747a09a1e8747afa51aaae1ac99685a25a688"}
May 28 15:07:46.811618 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.811591    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8nt5" event={"ID":"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc","Type":"ContainerStarted","Data":"6c7cb73041b241133f1233c5370f89a8a8edce6d39c15e8164ae646fdd115f2a"}
May 28 15:07:46.812787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.812768    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w6hk4" event={"ID":"5ecd7b4a-2ba1-4753-ba17-959add9303eb","Type":"ContainerStarted","Data":"8277c68538bb2d76cb233851814989b9726b84b1b1c27d8234eb6aa04a5de16b"}
May 28 15:07:46.815967 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.815921    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-61.ec2.internal" podStartSLOduration=1.815907493 podStartE2EDuration="1.815907493s" podCreationTimestamp="2026-05-28 15:07:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:07:46.815161011 +0000 UTC m=+3.600311672" watchObservedRunningTime="2026-05-28 15:07:46.815907493 +0000 UTC m=+3.601058154"
May 28 15:07:46.817442 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:46.817418    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wgnqk" event={"ID":"8f283641-0611-4d23-b85e-28cc5090beb1","Type":"ContainerStarted","Data":"25a75053c24ef7640bfbc8df06b088ddfb37177ae26071685a70ce5406b668de"}
May 28 15:07:47.311095 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:47.310815    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:47.311269 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:47.311169    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:47.311269 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:47.311233    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs podName:5086afac-ec94-48f4-8180-672970697414 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:49.311214292 +0000 UTC m=+6.096364935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs") pod "network-metrics-daemon-8jlgg" (UID: "5086afac-ec94-48f4-8180-672970697414") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:47.411949 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:47.411912    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:47.412198 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:47.412164    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:07:47.412276 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:47.412207    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:07:47.412276 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:47.412221    2560 projected.go:194] Error preparing data for projected volume kube-api-access-hf5br for pod openshift-network-diagnostics/network-check-target-rmtwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:47.412380 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:47.412299    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br podName:237da9f1-53ff-4878-9254-e0a1cc9a01db nodeName:}" failed. No retries permitted until 2026-05-28 15:07:49.412278795 +0000 UTC m=+6.197429437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf5br" (UniqueName: "kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br") pod "network-check-target-rmtwg" (UID: "237da9f1-53ff-4878-9254-e0a1cc9a01db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:47.783598 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:47.783428    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:47.783598 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:47.783576    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:07:47.784091 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:47.784064    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:47.784188 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:47.784154    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:07:47.862378 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:47.861953    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal" event={"ID":"1715a01d738f9b219bcc1d2d2749b38f","Type":"ContainerStarted","Data":"2a1c4adced068698ea507f55849b384c074b90870a2ff1ab211d58628de94fb4"}
May 28 15:07:47.877945 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:47.877895    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-61.ec2.internal" podStartSLOduration=2.877877367 podStartE2EDuration="2.877877367s" podCreationTimestamp="2026-05-28 15:07:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:07:47.877732614 +0000 UTC m=+4.662883276" watchObservedRunningTime="2026-05-28 15:07:47.877877367 +0000 UTC m=+4.663028021"
May 28 15:07:49.326535 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:49.326497    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:49.326990 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:49.326649    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:49.326990 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:49.326718    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs podName:5086afac-ec94-48f4-8180-672970697414 nodeName:}" failed. No retries permitted until 2026-05-28 15:07:53.326697649 +0000 UTC m=+10.111848289 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs") pod "network-metrics-daemon-8jlgg" (UID: "5086afac-ec94-48f4-8180-672970697414") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:49.426901 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:49.426859    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:49.427098 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:49.427067    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:07:49.427098 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:49.427093    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:07:49.427225 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:49.427108    2560 projected.go:194] Error preparing data for projected volume kube-api-access-hf5br for pod openshift-network-diagnostics/network-check-target-rmtwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:49.427225 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:49.427170    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br podName:237da9f1-53ff-4878-9254-e0a1cc9a01db nodeName:}" failed. No retries permitted until 2026-05-28 15:07:53.42715108 +0000 UTC m=+10.212301734 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf5br" (UniqueName: "kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br") pod "network-check-target-rmtwg" (UID: "237da9f1-53ff-4878-9254-e0a1cc9a01db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:49.780777 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:49.780697    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:49.780940 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:49.780841    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:07:49.781297 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:49.781278    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:49.781406 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:49.781373    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:07:51.782009 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:51.780420    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:51.782009 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:51.781655    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:07:51.782009 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:51.781741    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:51.782009 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:51.781870    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:07:53.362602 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:53.362548    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:53.363056 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:53.362709    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:53.363056 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:53.362777    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs podName:5086afac-ec94-48f4-8180-672970697414 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:01.362757535 +0000 UTC m=+18.147908174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs") pod "network-metrics-daemon-8jlgg" (UID: "5086afac-ec94-48f4-8180-672970697414") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:07:53.463931 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:53.463872    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:53.464089 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:53.464061    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:07:53.464089 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:53.464080    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:07:53.464240 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:53.464091    2560 projected.go:194] Error preparing data for projected volume kube-api-access-hf5br for pod openshift-network-diagnostics/network-check-target-rmtwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:53.464240 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:53.464148    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br podName:237da9f1-53ff-4878-9254-e0a1cc9a01db nodeName:}" failed. No retries permitted until 2026-05-28 15:08:01.464128122 +0000 UTC m=+18.249278773 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf5br" (UniqueName: "kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br") pod "network-check-target-rmtwg" (UID: "237da9f1-53ff-4878-9254-e0a1cc9a01db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:07:53.781643 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:53.781537    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:53.781643 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:53.781569    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:53.781802 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:53.781656    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:07:53.782045 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:53.782023    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:07:55.780570 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:55.780461    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:55.780570 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:55.780509    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:55.781128 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:55.780608    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:07:55.781128 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:55.780679    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:07:57.780965 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:57.780924    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:57.780965 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:57.780945    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:57.781514 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:57.781049    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:07:57.781514 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:57.781168    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:07:59.780482 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:59.780448    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:07:59.780946 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:07:59.780575    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:07:59.780946 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:59.780579    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:07:59.780946 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:07:59.780674    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:01.419571 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:01.419529    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:01.420097 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:01.419714    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:08:01.420097 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:01.419776    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs podName:5086afac-ec94-48f4-8180-672970697414 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:17.41975805 +0000 UTC m=+34.204908687 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs") pod "network-metrics-daemon-8jlgg" (UID: "5086afac-ec94-48f4-8180-672970697414") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:08:01.520225 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:01.520184    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:01.520397 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:01.520357    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:08:01.520397 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:01.520383    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:08:01.520397 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:01.520395    2560 projected.go:194] Error preparing data for projected volume kube-api-access-hf5br for pod openshift-network-diagnostics/network-check-target-rmtwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:08:01.520514 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:01.520452    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br podName:237da9f1-53ff-4878-9254-e0a1cc9a01db nodeName:}" failed. No retries permitted until 2026-05-28 15:08:17.520437235 +0000 UTC m=+34.305587873 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf5br" (UniqueName: "kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br") pod "network-check-target-rmtwg" (UID: "237da9f1-53ff-4878-9254-e0a1cc9a01db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:08:01.781071 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:01.780985    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:01.781219 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:01.781118    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:01.781219 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:01.781175    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:01.781332 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:01.781262    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:03.781732 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:03.781696    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:03.782136 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:03.781746    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:03.782136 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:03.781828    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:03.782136 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:03.781952    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:05.780607 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.780427    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:05.781205 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.780458    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:05.781205 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:05.780698    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:05.781205 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:05.780811    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:05.895066 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.895031    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w6hk4" event={"ID":"5ecd7b4a-2ba1-4753-ba17-959add9303eb","Type":"ContainerStarted","Data":"bbafff797a6a240010f4ab794375699808ec91a8fd2f48faf18b26b533e59fe7"}
May 28 15:08:05.896435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.896402    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wgnqk" event={"ID":"8f283641-0611-4d23-b85e-28cc5090beb1","Type":"ContainerStarted","Data":"facf94349ddf04a61db81f9fe08e047090a4490986a5df312764e65ddca2ab3a"}
May 28 15:08:05.897689 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.897666    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw" event={"ID":"1e40d747-fabc-4dec-9063-277ba893e6b7","Type":"ContainerStarted","Data":"cec177d45e533f411e21637ac47b5af25741a8d374b71a8104989b0737e52b48"}
May 28 15:08:05.898989 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.898968    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g6dcp" event={"ID":"757cacce-776f-4504-94fe-de28576cf4b4","Type":"ContainerStarted","Data":"97fd403fac18e0ca29d193191eaa84bcbbb2decbb3225cc89f03c837b6d721bc"}
May 28 15:08:05.900169 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.900148    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-46rrp" event={"ID":"a9714d3f-962d-4437-a7dc-c3f1322109b3","Type":"ContainerStarted","Data":"2df05b40d761e45b6d90058e9beddc436e10b66ff9d4002cc2b7746670adb37c"}
May 28 15:08:05.901329 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.901310    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4bxkw" event={"ID":"1aea7a9e-2599-4468-9bec-d293c48f84a9","Type":"ContainerStarted","Data":"e09d0af7046a766452bb3dd011e68f3c6f1c9d674b224d4a0e5f076231c2cca9"}
May 28 15:08:05.902656 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.902631    2560 generic.go:358] "Generic (PLEG): container finished" podID="30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc" containerID="410c284866a8d19bf0c7ac9b285c54bc3f3ed1b6bee4a4886f37b83c77b4a7d0" exitCode=0
May 28 15:08:05.902725 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.902670    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8nt5" event={"ID":"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc","Type":"ContainerDied","Data":"410c284866a8d19bf0c7ac9b285c54bc3f3ed1b6bee4a4886f37b83c77b4a7d0"}
May 28 15:08:05.910108 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.910072    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w6hk4" podStartSLOduration=4.4458112 podStartE2EDuration="22.910061612s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:07:46.421316102 +0000 UTC m=+3.206466739" lastFinishedPulling="2026-05-28 15:08:04.885566495 +0000 UTC m=+21.670717151" observedRunningTime="2026-05-28 15:08:05.909819916 +0000 UTC m=+22.694970576" watchObservedRunningTime="2026-05-28 15:08:05.910061612 +0000 UTC m=+22.695212272"
May 28 15:08:05.943526 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.943441    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-46rrp" podStartSLOduration=4.412037413 podStartE2EDuration="22.94342973s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:07:46.421386264 +0000 UTC m=+3.206536905" lastFinishedPulling="2026-05-28 15:08:04.952778578 +0000 UTC m=+21.737929222" observedRunningTime="2026-05-28 15:08:05.926911481 +0000 UTC m=+22.712062142" watchObservedRunningTime="2026-05-28 15:08:05.94342973 +0000 UTC m=+22.728580428"
May 28 15:08:05.943659 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.943524    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-wgnqk" podStartSLOduration=4.40657361 podStartE2EDuration="22.943520057s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:07:46.421171765 +0000 UTC m=+3.206322405" lastFinishedPulling="2026-05-28 15:08:04.95811821 +0000 UTC m=+21.743268852" observedRunningTime="2026-05-28 15:08:05.943220521 +0000 UTC m=+22.728371181" watchObservedRunningTime="2026-05-28 15:08:05.943520057 +0000 UTC m=+22.728670716"
May 28 15:08:05.956570 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.956520    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g6dcp" podStartSLOduration=4.481778823 podStartE2EDuration="22.956508792s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:07:46.410838594 +0000 UTC m=+3.195989232" lastFinishedPulling="2026-05-28 15:08:04.885568551 +0000 UTC m=+21.670719201" observedRunningTime="2026-05-28 15:08:05.956111381 +0000 UTC m=+22.741262038" watchObservedRunningTime="2026-05-28 15:08:05.956508792 +0000 UTC m=+22.741659474"
May 28 15:08:05.980705 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:05.980658    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4bxkw" podStartSLOduration=4.444724627 podStartE2EDuration="22.980645479s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:07:46.41723337 +0000 UTC m=+3.202384011" lastFinishedPulling="2026-05-28 15:08:04.953154225 +0000 UTC m=+21.738304863" observedRunningTime="2026-05-28 15:08:05.980214112 +0000 UTC m=+22.765364773" watchObservedRunningTime="2026-05-28 15:08:05.980645479 +0000 UTC m=+22.765796139"
May 28 15:08:06.442457 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.442302    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 15:08:06.758839 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.758716    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T15:08:06.442453312Z","UUID":"5c841aa1-7032-4f26-a5ce-42e5fba43ab3","Handler":null,"Name":"","Endpoint":""}
May 28 15:08:06.761596 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.761542    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 15:08:06.761596 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.761601    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 15:08:06.908869 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.908826    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" event={"ID":"5344fe70-83db-47a2-bf7f-fe4b0659ba27","Type":"ContainerStarted","Data":"c3d5e9808b6f1a15e8e7f0cdaadb6a544559b16ed0691ea208326c907da9681a"}
May 28 15:08:06.908869 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.908872    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" event={"ID":"5344fe70-83db-47a2-bf7f-fe4b0659ba27","Type":"ContainerStarted","Data":"8de0ecb3e17fb24ac733c333506e59f7a347e9c9001ea1060f8aa5838373d3e6"}
May 28 15:08:06.909316 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.908882    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" event={"ID":"5344fe70-83db-47a2-bf7f-fe4b0659ba27","Type":"ContainerStarted","Data":"e74e2dcb5360510ee2523c229d44bb25173a86320edd20d46f29856321d5daf1"}
May 28 15:08:06.909316 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.908891    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" event={"ID":"5344fe70-83db-47a2-bf7f-fe4b0659ba27","Type":"ContainerStarted","Data":"d873d56d505b7d0fe0abe321933f31c43511f0c98417a095a65b91cd1a6e80e1"}
May 28 15:08:06.909316 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.908904    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" event={"ID":"5344fe70-83db-47a2-bf7f-fe4b0659ba27","Type":"ContainerStarted","Data":"24441a25bf8ec1588a33be2cadd86d7f1ed498f7e18cedf3af410294d3aad90f"}
May 28 15:08:06.909316 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.908913    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" event={"ID":"5344fe70-83db-47a2-bf7f-fe4b0659ba27","Type":"ContainerStarted","Data":"eb1aecbefadc8b5c9d8216963de1d9c2cd24ac37431da77ae9d72633c7684c7a"}
May 28 15:08:06.910869 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.910824    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw" event={"ID":"1e40d747-fabc-4dec-9063-277ba893e6b7","Type":"ContainerStarted","Data":"a32f7b23f7cec9822709471de213724ad01488bc7a313c17cf8a586167c492ad"}
May 28 15:08:06.912679 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.912648    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t9f8f" event={"ID":"f1abad2b-d3f0-474f-a60a-cfa07f525595","Type":"ContainerStarted","Data":"246a41541d41dc9eba7f4cf30a59224d86178cd35125a64bdcbe376a52e7fd85"}
May 28 15:08:06.926827 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:06.926785    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t9f8f" podStartSLOduration=5.390443413 podStartE2EDuration="23.926773308s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:07:46.417173937 +0000 UTC m=+3.202324579" lastFinishedPulling="2026-05-28 15:08:04.95350383 +0000 UTC m=+21.738654474" observedRunningTime="2026-05-28 15:08:06.926419524 +0000 UTC m=+23.711570185" watchObservedRunningTime="2026-05-28 15:08:06.926773308 +0000 UTC m=+23.711923985"
May 28 15:08:07.040588 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:07.040476    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:08:07.041346 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:07.041324    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:08:07.780727 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:07.780643    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:07.780891 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:07.780740    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:07.780981 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:07.780644    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:07.781108 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:07.781089    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:07.916810 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:07.916768    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw" event={"ID":"1e40d747-fabc-4dec-9063-277ba893e6b7","Type":"ContainerStarted","Data":"f2431b96920fa5b44114a42106029a5a59c2eae495c562ee845d43d8520130ca"}
May 28 15:08:08.918975 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:08.918942    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 15:08:09.780486 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:09.780446    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:09.780677 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:09.780614    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:09.780814 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:09.780789    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:09.780924 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:09.780896    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:10.923861 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:10.923830    2560 generic.go:358] "Generic (PLEG): container finished" podID="30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc" containerID="7911aa870f843030efa798ee638ea577e74937a0d882291f1f7bf00c0df5d9e3" exitCode=0
May 28 15:08:10.924389 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:10.923921    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8nt5" event={"ID":"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc","Type":"ContainerDied","Data":"7911aa870f843030efa798ee638ea577e74937a0d882291f1f7bf00c0df5d9e3"}
May 28 15:08:10.927009 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:10.926991    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" event={"ID":"5344fe70-83db-47a2-bf7f-fe4b0659ba27","Type":"ContainerStarted","Data":"f7689524b63a502ebf8fa13e1ad96619c230e171f2420565d8654002b29ae77f"}
May 28 15:08:10.947348 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:10.947311    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hv6hw" podStartSLOduration=6.89298932 podStartE2EDuration="27.947300043s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:07:46.411542432 +0000 UTC m=+3.196693071" lastFinishedPulling="2026-05-28 15:08:07.465853156 +0000 UTC m=+24.251003794" observedRunningTime="2026-05-28 15:08:07.937430668 +0000 UTC m=+24.722581331" watchObservedRunningTime="2026-05-28 15:08:10.947300043 +0000 UTC m=+27.732450703"
May 28 15:08:11.623667 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:11.623427    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:08:11.623840 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:11.623795    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 15:08:11.624185 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:11.624161    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w6hk4"
May 28 15:08:11.780531 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:11.780508    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:11.780642 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:11.780508    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:11.780705 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:11.780640    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:11.780705 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:11.780674    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:11.931105 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:11.930997    2560 generic.go:358] "Generic (PLEG): container finished" podID="30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc" containerID="f0dd69ff28ff8f63a27acc2a5ff2567ec30517a83165f5ac67623a189ce4835c" exitCode=0
May 28 15:08:11.931105 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:11.931077    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8nt5" event={"ID":"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc","Type":"ContainerDied","Data":"f0dd69ff28ff8f63a27acc2a5ff2567ec30517a83165f5ac67623a189ce4835c"}
May 28 15:08:12.935395 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:12.935360    2560 generic.go:358] "Generic (PLEG): container finished" podID="30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc" containerID="f92d09cada78ffcf327951de108927d0241fd8c7dff49ad6065901ffab987868" exitCode=0
May 28 15:08:12.935923 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:12.935436    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8nt5" event={"ID":"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc","Type":"ContainerDied","Data":"f92d09cada78ffcf327951de108927d0241fd8c7dff49ad6065901ffab987868"}
May 28 15:08:12.938860 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:12.938835    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" event={"ID":"5344fe70-83db-47a2-bf7f-fe4b0659ba27","Type":"ContainerStarted","Data":"cb64f46e27d1cde6817b60dff60933b51547a6cdf12e763fa8112fb7e218aba0"}
May 28 15:08:12.939216 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:12.939191    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:08:12.954987 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:12.954966    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:08:12.982794 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:12.982749    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x" podStartSLOduration=10.681121609 podStartE2EDuration="29.982735157s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:07:46.421420202 +0000 UTC m=+3.206570855" lastFinishedPulling="2026-05-28 15:08:05.723033761 +0000 UTC m=+22.508184403" observedRunningTime="2026-05-28 15:08:12.982616882 +0000 UTC m=+29.767767541" watchObservedRunningTime="2026-05-28 15:08:12.982735157 +0000 UTC m=+29.767885857"
May 28 15:08:13.781328 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:13.781163    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:13.781503 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:13.781202    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:13.781503 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:13.781389    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:13.781625 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:13.781518    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:13.941539 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:13.941503    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:08:13.941539 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:13.941537    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:08:13.959377 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:13.959353    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:08:14.509116 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:14.508534    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rmtwg"]
May 28 15:08:14.509116 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:14.508666    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:14.509116 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:14.508767    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:14.510686 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:14.510662    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8jlgg"]
May 28 15:08:14.510912 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:14.510757    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:14.510912 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:14.510864    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:15.780846 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:15.780807    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:15.781520 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:15.780965    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:16.780784 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:16.780753    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:16.780931 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:16.780862    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:17.439878 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:17.439845    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:17.440097 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:17.439987    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:08:17.440097 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:17.440049    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs podName:5086afac-ec94-48f4-8180-672970697414 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:49.440033874 +0000 UTC m=+66.225184512 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs") pod "network-metrics-daemon-8jlgg" (UID: "5086afac-ec94-48f4-8180-672970697414") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:08:17.540396 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:17.540363    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:17.540640 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:17.540499    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:08:17.540640 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:17.540516    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:08:17.540640 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:17.540525    2560 projected.go:194] Error preparing data for projected volume kube-api-access-hf5br for pod openshift-network-diagnostics/network-check-target-rmtwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:08:17.540640 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:17.540589    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br podName:237da9f1-53ff-4878-9254-e0a1cc9a01db nodeName:}" failed. No retries permitted until 2026-05-28 15:08:49.540576229 +0000 UTC m=+66.325726867 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hf5br" (UniqueName: "kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br") pod "network-check-target-rmtwg" (UID: "237da9f1-53ff-4878-9254-e0a1cc9a01db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:08:17.780673 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:17.780599    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:17.780894 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:17.780756    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jlgg" podUID="5086afac-ec94-48f4-8180-672970697414"
May 28 15:08:18.780805 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:18.780742    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:18.781193 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:18.780841    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rmtwg" podUID="237da9f1-53ff-4878-9254-e0a1cc9a01db"
May 28 15:08:19.047584 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.047539    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-61.ec2.internal" event="NodeReady"
May 28 15:08:19.047695 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.047680    2560 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 15:08:19.092809 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.092787    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vs6bq"]
May 28 15:08:19.124974 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.124950    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nwwjb"]
May 28 15:08:19.125173 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.125156    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.128881 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.128839    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b5hqh\""
May 28 15:08:19.129325 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.129302    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 15:08:19.129885 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.129859    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 15:08:19.139084 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.139064    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vs6bq"]
May 28 15:08:19.139159 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.139089    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwwjb"]
May 28 15:08:19.139203 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.139167    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:19.142191 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.142035    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 15:08:19.142191 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.142118    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 15:08:19.142351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.142194    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 15:08:19.142408 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.142374    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n5mwm\""
May 28 15:08:19.254397 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.254360    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-config-volume\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.254521 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.254414    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:19.254521 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.254438    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k8k6\" (UniqueName: \"kubernetes.io/projected/ae300cc7-3fd0-4422-816d-eec9479c9a0b-kube-api-access-9k8k6\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:19.254521 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.254475    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.254521 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.254508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-695jh\" (UniqueName: \"kubernetes.io/projected/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-kube-api-access-695jh\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.254698 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.254547    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-tmp-dir\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.355199 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.355115    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-tmp-dir\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.355199 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.355164    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-config-volume\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.355401 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.355202    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:19.355401 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.355219    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9k8k6\" (UniqueName: \"kubernetes.io/projected/ae300cc7-3fd0-4422-816d-eec9479c9a0b-kube-api-access-9k8k6\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:19.355401 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.355245    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.355401 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.355272    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-695jh\" (UniqueName: \"kubernetes.io/projected/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-kube-api-access-695jh\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.355401 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:19.355346    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:08:19.355618 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:19.355412    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:08:19.355618 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:19.355440    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert podName:ae300cc7-3fd0-4422-816d-eec9479c9a0b nodeName:}" failed. No retries permitted until 2026-05-28 15:08:19.855420453 +0000 UTC m=+36.640571093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert") pod "ingress-canary-nwwjb" (UID: "ae300cc7-3fd0-4422-816d-eec9479c9a0b") : secret "canary-serving-cert" not found
May 28 15:08:19.355618 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:19.355495    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls podName:cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:19.855467209 +0000 UTC m=+36.640617860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls") pod "dns-default-vs6bq" (UID: "cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902") : secret "dns-default-metrics-tls" not found
May 28 15:08:19.355618 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.355523    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-tmp-dir\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.355788 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.355772    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-config-volume\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.366250 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.366096    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-695jh\" (UniqueName: \"kubernetes.io/projected/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-kube-api-access-695jh\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.381079 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.381054    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k8k6\" (UniqueName: \"kubernetes.io/projected/ae300cc7-3fd0-4422-816d-eec9479c9a0b-kube-api-access-9k8k6\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:19.781445 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.781338    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:19.784519 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.784495    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8wjkp\""
May 28 15:08:19.784519 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.784516    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 15:08:19.858319 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.858285    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:19.858319 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.858326    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:19.858580 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:19.858417    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:08:19.858580 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:19.858461    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls podName:cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:20.858449444 +0000 UTC m=+37.643600082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls") pod "dns-default-vs6bq" (UID: "cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902") : secret "dns-default-metrics-tls" not found
May 28 15:08:19.858715 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:19.858586    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:08:19.858715 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:19.858647    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert podName:ae300cc7-3fd0-4422-816d-eec9479c9a0b nodeName:}" failed. No retries permitted until 2026-05-28 15:08:20.858633534 +0000 UTC m=+37.643784185 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert") pod "ingress-canary-nwwjb" (UID: "ae300cc7-3fd0-4422-816d-eec9479c9a0b") : secret "canary-serving-cert" not found
May 28 15:08:19.954478 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.954444    2560 generic.go:358] "Generic (PLEG): container finished" podID="30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc" containerID="2944a1438d27cca0b807bad66caaa759bdfb05ecf3e4523b1fbbf8926fd8370c" exitCode=0
May 28 15:08:19.954662 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:19.954508    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8nt5" event={"ID":"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc","Type":"ContainerDied","Data":"2944a1438d27cca0b807bad66caaa759bdfb05ecf3e4523b1fbbf8926fd8370c"}
May 28 15:08:20.780826 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:20.780792    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:20.783872 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:20.783844    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 15:08:20.785053 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:20.785033    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 15:08:20.785053 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:20.785045    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xkj5\""
May 28 15:08:20.866850 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:20.866821    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:20.866981 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:20.866862    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:20.866981 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:20.866966    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:08:20.867066 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:20.867026    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert podName:ae300cc7-3fd0-4422-816d-eec9479c9a0b nodeName:}" failed. No retries permitted until 2026-05-28 15:08:22.867009808 +0000 UTC m=+39.652160450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert") pod "ingress-canary-nwwjb" (UID: "ae300cc7-3fd0-4422-816d-eec9479c9a0b") : secret "canary-serving-cert" not found
May 28 15:08:20.867066 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:20.867035    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:08:20.867148 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:20.867124    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls podName:cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:22.867109692 +0000 UTC m=+39.652260330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls") pod "dns-default-vs6bq" (UID: "cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902") : secret "dns-default-metrics-tls" not found
May 28 15:08:20.959069 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:20.959032    2560 generic.go:358] "Generic (PLEG): container finished" podID="30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc" containerID="274912cf6f937882125eaa2fea65c0a850817945ccfd9ecb1cd5dcf24e886222" exitCode=0
May 28 15:08:20.959228 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:20.959089    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8nt5" event={"ID":"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc","Type":"ContainerDied","Data":"274912cf6f937882125eaa2fea65c0a850817945ccfd9ecb1cd5dcf24e886222"}
May 28 15:08:21.963160 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:21.963129    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8nt5" event={"ID":"30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc","Type":"ContainerStarted","Data":"c9bf326e2afe730f2ac6043bf09a69dcf096eb0ef36cbc5bf7364176433cafc1"}
May 28 15:08:21.989128 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:21.989079    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q8nt5" podStartSLOduration=6.4033212299999995 podStartE2EDuration="38.989067223s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:07:46.408108477 +0000 UTC m=+3.193259115" lastFinishedPulling="2026-05-28 15:08:18.993854466 +0000 UTC m=+35.779005108" observedRunningTime="2026-05-28 15:08:21.986144173 +0000 UTC m=+38.771294833" watchObservedRunningTime="2026-05-28 15:08:21.989067223 +0000 UTC m=+38.774217883"
May 28 15:08:22.881934 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:22.881894    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:22.881934 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:22.881942    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:22.882142 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:22.882037    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:08:22.882142 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:22.882041    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:08:22.882142 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:22.882088    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls podName:cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:26.882075671 +0000 UTC m=+43.667226309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls") pod "dns-default-vs6bq" (UID: "cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902") : secret "dns-default-metrics-tls" not found
May 28 15:08:22.882142 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:22.882108    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert podName:ae300cc7-3fd0-4422-816d-eec9479c9a0b nodeName:}" failed. No retries permitted until 2026-05-28 15:08:26.882093857 +0000 UTC m=+43.667244495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert") pod "ingress-canary-nwwjb" (UID: "ae300cc7-3fd0-4422-816d-eec9479c9a0b") : secret "canary-serving-cert" not found
May 28 15:08:26.911015 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:26.910966    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:26.911478 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:26.911024    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:26.911478 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:26.911112    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:08:26.911478 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:26.911128    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:08:26.911478 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:26.911175    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert podName:ae300cc7-3fd0-4422-816d-eec9479c9a0b nodeName:}" failed. No retries permitted until 2026-05-28 15:08:34.911160901 +0000 UTC m=+51.696311539 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert") pod "ingress-canary-nwwjb" (UID: "ae300cc7-3fd0-4422-816d-eec9479c9a0b") : secret "canary-serving-cert" not found
May 28 15:08:26.911478 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:26.911189    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls podName:cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:34.911182955 +0000 UTC m=+51.696333592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls") pod "dns-default-vs6bq" (UID: "cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902") : secret "dns-default-metrics-tls" not found
May 28 15:08:34.960715 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:34.960669    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:34.961228 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:34.960733    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:34.961228 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:34.960828    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:08:34.961228 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:34.960847    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:08:34.961228 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:34.960905    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert podName:ae300cc7-3fd0-4422-816d-eec9479c9a0b nodeName:}" failed. No retries permitted until 2026-05-28 15:08:50.960885132 +0000 UTC m=+67.746035770 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert") pod "ingress-canary-nwwjb" (UID: "ae300cc7-3fd0-4422-816d-eec9479c9a0b") : secret "canary-serving-cert" not found
May 28 15:08:34.961228 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:34.960919    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls podName:cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902 nodeName:}" failed. No retries permitted until 2026-05-28 15:08:50.960912859 +0000 UTC m=+67.746063497 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls") pod "dns-default-vs6bq" (UID: "cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902") : secret "dns-default-metrics-tls" not found
May 28 15:08:45.956063 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:45.956033    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2r4x"
May 28 15:08:47.377614 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.377581    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"]
May 28 15:08:47.381592 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.381571    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"
May 28 15:08:47.384409 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.384387    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 28 15:08:47.384568 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.384416    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4kd2s\""
May 28 15:08:47.384568 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.384462    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 28 15:08:47.384568 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.384527    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 28 15:08:47.385813 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.385792    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 28 15:08:47.388253 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.387855    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"]
May 28 15:08:47.394859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.394841    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"]
May 28 15:08:47.396521 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.396508    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.399121 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.399100    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 28 15:08:47.399203 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.399138    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 28 15:08:47.399203 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.399147    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 28 15:08:47.399203 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.399177    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 28 15:08:47.408902 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.408886    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"]
May 28 15:08:47.547415 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.547369    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-hub\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.547599 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.547428    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4285q\" (UniqueName: \"kubernetes.io/projected/9b326cd2-e9ce-4a7d-a0ee-37c2303809d0-kube-api-access-4285q\") pod \"managed-serviceaccount-addon-agent-74b4466d54-rw88q\" (UID: \"9b326cd2-e9ce-4a7d-a0ee-37c2303809d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"
May 28 15:08:47.547599 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.547478    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.547599 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.547510    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-ca\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.547599 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.547527    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.547599 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.547545    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-725rp\" (UniqueName: \"kubernetes.io/projected/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-kube-api-access-725rp\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.547599 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.547587    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b326cd2-e9ce-4a7d-a0ee-37c2303809d0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74b4466d54-rw88q\" (UID: \"9b326cd2-e9ce-4a7d-a0ee-37c2303809d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"
May 28 15:08:47.547787 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.547603    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.648050 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.647956    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.648050 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.648027    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-ca\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.648251 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.648054    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.648251 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.648082    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-725rp\" (UniqueName: \"kubernetes.io/projected/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-kube-api-access-725rp\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.648251 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.648108    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b326cd2-e9ce-4a7d-a0ee-37c2303809d0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74b4466d54-rw88q\" (UID: \"9b326cd2-e9ce-4a7d-a0ee-37c2303809d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"
May 28 15:08:47.648251 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.648129    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.648251 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.648189    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-hub\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.648251 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.648219    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4285q\" (UniqueName: \"kubernetes.io/projected/9b326cd2-e9ce-4a7d-a0ee-37c2303809d0-kube-api-access-4285q\") pod \"managed-serviceaccount-addon-agent-74b4466d54-rw88q\" (UID: \"9b326cd2-e9ce-4a7d-a0ee-37c2303809d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"
May 28 15:08:47.648910 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.648883    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.651291 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.651261    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-hub\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.651443 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.651420    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b326cd2-e9ce-4a7d-a0ee-37c2303809d0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74b4466d54-rw88q\" (UID: \"9b326cd2-e9ce-4a7d-a0ee-37c2303809d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"
May 28 15:08:47.651488 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.651453    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.651593 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.651576    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.651635 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.651575    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-ca\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.657719 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.657697    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-725rp\" (UniqueName: \"kubernetes.io/projected/1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95-kube-api-access-725rp\") pod \"cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg\" (UID: \"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.657979 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.657960    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4285q\" (UniqueName: \"kubernetes.io/projected/9b326cd2-e9ce-4a7d-a0ee-37c2303809d0-kube-api-access-4285q\") pod \"managed-serviceaccount-addon-agent-74b4466d54-rw88q\" (UID: \"9b326cd2-e9ce-4a7d-a0ee-37c2303809d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"
May 28 15:08:47.702090 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.702062    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"
May 28 15:08:47.708820 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.708799    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:08:47.838456 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.838294    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q"]
May 28 15:08:47.841074 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:08:47.841049    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b326cd2_e9ce_4a7d_a0ee_37c2303809d0.slice/crio-6369fb8ac2ad84a09bf27e855a908a6f1ccb41eda071c91d8dbb1198be1a6606 WatchSource:0}: Error finding container 6369fb8ac2ad84a09bf27e855a908a6f1ccb41eda071c91d8dbb1198be1a6606: Status 404 returned error can't find the container with id 6369fb8ac2ad84a09bf27e855a908a6f1ccb41eda071c91d8dbb1198be1a6606
May 28 15:08:47.855495 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:47.855472    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"]
May 28 15:08:47.857833 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:08:47.857806    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b1f4b61_e0a7_4b31_b890_8e5c84e0ea95.slice/crio-16bceea98a587bd772c052200dea6d5dc46fc49e523b6c4ed546eda5a90da6ad WatchSource:0}: Error finding container 16bceea98a587bd772c052200dea6d5dc46fc49e523b6c4ed546eda5a90da6ad: Status 404 returned error can't find the container with id 16bceea98a587bd772c052200dea6d5dc46fc49e523b6c4ed546eda5a90da6ad
May 28 15:08:48.008484 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:48.008385    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q" event={"ID":"9b326cd2-e9ce-4a7d-a0ee-37c2303809d0","Type":"ContainerStarted","Data":"6369fb8ac2ad84a09bf27e855a908a6f1ccb41eda071c91d8dbb1198be1a6606"}
May 28 15:08:48.009467 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:48.009440    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" event={"ID":"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95","Type":"ContainerStarted","Data":"16bceea98a587bd772c052200dea6d5dc46fc49e523b6c4ed546eda5a90da6ad"}
May 28 15:08:49.461680 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:49.461639    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:08:49.464735 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:49.464706    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 15:08:49.472002 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:49.471983    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 15:08:49.472111 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:49.472049    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs podName:5086afac-ec94-48f4-8180-672970697414 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:53.472026819 +0000 UTC m=+130.257177462 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs") pod "network-metrics-daemon-8jlgg" (UID: "5086afac-ec94-48f4-8180-672970697414") : secret "metrics-daemon-secret" not found
May 28 15:08:49.562906 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:49.562860    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:49.566064 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:49.566034    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 15:08:49.575642 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:49.575616    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 15:08:49.587360 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:49.586916    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf5br\" (UniqueName: \"kubernetes.io/projected/237da9f1-53ff-4878-9254-e0a1cc9a01db-kube-api-access-hf5br\") pod \"network-check-target-rmtwg\" (UID: \"237da9f1-53ff-4878-9254-e0a1cc9a01db\") " pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:49.591948 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:49.591926    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6xkj5\""
May 28 15:08:49.600171 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:49.600146    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:50.973813 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:50.973772    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:08:50.974367 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:50.973844    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:08:50.974367 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:50.973974    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:08:50.974367 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:50.973985    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:08:50.974367 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:50.974038    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls podName:cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902 nodeName:}" failed. No retries permitted until 2026-05-28 15:09:22.974018381 +0000 UTC m=+99.759169056 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls") pod "dns-default-vs6bq" (UID: "cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902") : secret "dns-default-metrics-tls" not found
May 28 15:08:50.974367 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:08:50.974060    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert podName:ae300cc7-3fd0-4422-816d-eec9479c9a0b nodeName:}" failed. No retries permitted until 2026-05-28 15:09:22.974049481 +0000 UTC m=+99.759200119 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert") pod "ingress-canary-nwwjb" (UID: "ae300cc7-3fd0-4422-816d-eec9479c9a0b") : secret "canary-serving-cert" not found
May 28 15:08:52.421265 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:52.421238    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rmtwg"]
May 28 15:08:52.424976 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:08:52.424926    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237da9f1_53ff_4878_9254_e0a1cc9a01db.slice/crio-f17df61f12b2ab4d5492ed4eaf3396551aac60258c1e9f7b6077265c8f76a768 WatchSource:0}: Error finding container f17df61f12b2ab4d5492ed4eaf3396551aac60258c1e9f7b6077265c8f76a768: Status 404 returned error can't find the container with id f17df61f12b2ab4d5492ed4eaf3396551aac60258c1e9f7b6077265c8f76a768
May 28 15:08:53.023167 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:53.023120    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q" event={"ID":"9b326cd2-e9ce-4a7d-a0ee-37c2303809d0","Type":"ContainerStarted","Data":"371dcd20a176be6bde24dd6a0a7a76c85cdffac267d979ceac4fe390d878d63e"}
May 28 15:08:53.025282 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:53.024829    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" event={"ID":"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95","Type":"ContainerStarted","Data":"a589de4ea9b48d8cf56200bed493435d72899db2c2d1c36610c5b17e889f4f3a"}
May 28 15:08:53.026910 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:53.026885    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rmtwg" event={"ID":"237da9f1-53ff-4878-9254-e0a1cc9a01db","Type":"ContainerStarted","Data":"f17df61f12b2ab4d5492ed4eaf3396551aac60258c1e9f7b6077265c8f76a768"}
May 28 15:08:53.042514 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:53.042454    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74b4466d54-rw88q" podStartSLOduration=1.593745496 podStartE2EDuration="6.042439032s" podCreationTimestamp="2026-05-28 15:08:47 +0000 UTC" firstStartedPulling="2026-05-28 15:08:47.843205645 +0000 UTC m=+64.628356284" lastFinishedPulling="2026-05-28 15:08:52.291899163 +0000 UTC m=+69.077049820" observedRunningTime="2026-05-28 15:08:53.041692655 +0000 UTC m=+69.826843315" watchObservedRunningTime="2026-05-28 15:08:53.042439032 +0000 UTC m=+69.827589693"
May 28 15:08:56.034776 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:56.034691    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rmtwg" event={"ID":"237da9f1-53ff-4878-9254-e0a1cc9a01db","Type":"ContainerStarted","Data":"77ea90ef9ef91a006d4bf372df20513fbc334dd14d024297630440970a9af426"}
May 28 15:08:56.035130 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:56.034815    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:08:56.051149 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:56.051106    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rmtwg" podStartSLOduration=69.845595091 podStartE2EDuration="1m13.051092236s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:08:52.426882239 +0000 UTC m=+69.212032876" lastFinishedPulling="2026-05-28 15:08:55.632379383 +0000 UTC m=+72.417530021" observedRunningTime="2026-05-28 15:08:56.050951422 +0000 UTC m=+72.836102082" watchObservedRunningTime="2026-05-28 15:08:56.051092236 +0000 UTC m=+72.836242896"
May 28 15:08:57.038447 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:57.038363    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" event={"ID":"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95","Type":"ContainerStarted","Data":"be61873a960cd1fc17ed16cbecd03427d81f9b8720a160b70a75408d398f1cd7"}
May 28 15:08:57.038447 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:57.038407    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" event={"ID":"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95","Type":"ContainerStarted","Data":"19d40a00a2f54028afb8b5cb3ba2ac7cf7f189ba7c6fb6e250fe856f55e56964"}
May 28 15:08:57.059320 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:08:57.059278    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" podStartSLOduration=1.164098181 podStartE2EDuration="10.059266596s" podCreationTimestamp="2026-05-28 15:08:47 +0000 UTC" firstStartedPulling="2026-05-28 15:08:47.859449243 +0000 UTC m=+64.644599882" lastFinishedPulling="2026-05-28 15:08:56.754617654 +0000 UTC m=+73.539768297" observedRunningTime="2026-05-28 15:08:57.058844931 +0000 UTC m=+73.843995594" watchObservedRunningTime="2026-05-28 15:08:57.059266596 +0000 UTC m=+73.844417256"
May 28 15:09:22.998049 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:22.998008    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:09:22.998049 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:22.998053    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:09:22.998515 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:09:22.998140    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:09:22.998515 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:09:22.998173    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:09:22.998515 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:09:22.998192    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls podName:cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902 nodeName:}" failed. No retries permitted until 2026-05-28 15:10:26.99817752 +0000 UTC m=+163.783328159 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls") pod "dns-default-vs6bq" (UID: "cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902") : secret "dns-default-metrics-tls" not found
May 28 15:09:22.998515 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:09:22.998244    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert podName:ae300cc7-3fd0-4422-816d-eec9479c9a0b nodeName:}" failed. No retries permitted until 2026-05-28 15:10:26.998228455 +0000 UTC m=+163.783379094 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert") pod "ingress-canary-nwwjb" (UID: "ae300cc7-3fd0-4422-816d-eec9479c9a0b") : secret "canary-serving-cert" not found
May 28 15:09:27.040644 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:27.040612    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rmtwg"
May 28 15:09:42.908690 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:42.908662    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wgnqk_8f283641-0611-4d23-b85e-28cc5090beb1/dns-node-resolver/0.log"
May 28 15:09:43.708844 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:43.708818    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g6dcp_757cacce-776f-4504-94fe-de28576cf4b4/node-ca/0.log"
May 28 15:09:53.495472 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:53.495426    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:09:53.497751 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:53.497731    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5086afac-ec94-48f4-8180-672970697414-metrics-certs\") pod \"network-metrics-daemon-8jlgg\" (UID: \"5086afac-ec94-48f4-8180-672970697414\") " pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:09:53.693216 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:53.693186    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8wjkp\""
May 28 15:09:53.700507 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:53.700478    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jlgg"
May 28 15:09:53.823661 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:53.823633    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8jlgg"]
May 28 15:09:53.826447 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:09:53.826419    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5086afac_ec94_48f4_8180_672970697414.slice/crio-8c91ffb1a1b8caa7c587e52d0ed63fd9621ba68fb5385bc2f3da8d63c880389b WatchSource:0}: Error finding container 8c91ffb1a1b8caa7c587e52d0ed63fd9621ba68fb5385bc2f3da8d63c880389b: Status 404 returned error can't find the container with id 8c91ffb1a1b8caa7c587e52d0ed63fd9621ba68fb5385bc2f3da8d63c880389b
May 28 15:09:54.151283 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:54.151246    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8jlgg" event={"ID":"5086afac-ec94-48f4-8180-672970697414","Type":"ContainerStarted","Data":"8c91ffb1a1b8caa7c587e52d0ed63fd9621ba68fb5385bc2f3da8d63c880389b"}
May 28 15:09:55.155374 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:55.155340    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8jlgg" event={"ID":"5086afac-ec94-48f4-8180-672970697414","Type":"ContainerStarted","Data":"8a8ab3b8949d70ce426f7e6b12f586e1a6343d4dc0d7515b4e95e474f89cd2c4"}
May 28 15:09:55.155374 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:55.155375    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8jlgg" event={"ID":"5086afac-ec94-48f4-8180-672970697414","Type":"ContainerStarted","Data":"8f5676e6e98066a0f7f06c4f16f441e2345a97364e73edda1ec965c0f29d3e14"}
May 28 15:09:55.172654 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:09:55.172602    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8jlgg" podStartSLOduration=131.270277008 podStartE2EDuration="2m12.172588684s" podCreationTimestamp="2026-05-28 15:07:43 +0000 UTC" firstStartedPulling="2026-05-28 15:09:53.828135066 +0000 UTC m=+130.613285704" lastFinishedPulling="2026-05-28 15:09:54.73044674 +0000 UTC m=+131.515597380" observedRunningTime="2026-05-28 15:09:55.171453876 +0000 UTC m=+131.956604548" watchObservedRunningTime="2026-05-28 15:09:55.172588684 +0000 UTC m=+131.957739376"
May 28 15:10:01.815576 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.815515    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8rkr6"]
May 28 15:10:01.817760 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.817739    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:01.820180 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.820153    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 15:10:01.820351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.820338    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 15:10:01.821626 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.821604    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 15:10:01.821836 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.821638    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 15:10:01.821937 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.821640    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nrcdd\""
May 28 15:10:01.830349 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.830331    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8rkr6"]
May 28 15:10:01.921763 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.921727    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-94cdc59b5-h52qg"]
May 28 15:10:01.923519 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.923499    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:01.926312 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.926287    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 15:10:01.926447 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.926432    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 15:10:01.927471 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.927454    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 15:10:01.928000 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.927986    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7pvbk\""
May 28 15:10:01.931935 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.931913    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 15:10:01.940751 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.940729    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-94cdc59b5-h52qg"]
May 28 15:10:01.956885 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.956863    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7mmw\" (UniqueName: \"kubernetes.io/projected/30ab8040-39a2-48a1-8dbd-f543a30b379d-kube-api-access-j7mmw\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:01.957333 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.956903    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30ab8040-39a2-48a1-8dbd-f543a30b379d-data-volume\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:01.957496 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.957477    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30ab8040-39a2-48a1-8dbd-f543a30b379d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:01.957682 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.957666    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30ab8040-39a2-48a1-8dbd-f543a30b379d-crio-socket\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:01.957815 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.957801    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30ab8040-39a2-48a1-8dbd-f543a30b379d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:01.970569 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:01.970520    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-94cdc59b5-h52qg"]
May 28 15:10:01.970724 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:10:01.970698    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-7qnb5 registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-7qnb5 registry-certificates registry-tls trusted-ca]: context canceled" pod="openshift-image-registry/image-registry-94cdc59b5-h52qg" podUID="d1c52025-a560-49f8-8531-dbe401ff5485"
May 28 15:10:02.058947 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.058916    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30ab8040-39a2-48a1-8dbd-f543a30b379d-data-volume\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.059055 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.058956    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-registry-certificates\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.059055 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.058983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-installation-pull-secrets\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.059151 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059082    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30ab8040-39a2-48a1-8dbd-f543a30b379d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.059151 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059127    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30ab8040-39a2-48a1-8dbd-f543a30b379d-crio-socket\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.059151 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059145    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30ab8040-39a2-48a1-8dbd-f543a30b379d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.059293 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qnb5\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-kube-api-access-7qnb5\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.059293 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059214    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30ab8040-39a2-48a1-8dbd-f543a30b379d-crio-socket\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.059293 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059241    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30ab8040-39a2-48a1-8dbd-f543a30b379d-data-volume\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.059293 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059285    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1c52025-a560-49f8-8531-dbe401ff5485-ca-trust-extracted\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.059478 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059376    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-image-registry-private-configuration\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.059478 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059407    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-bound-sa-token\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.059478 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059435    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j7mmw\" (UniqueName: \"kubernetes.io/projected/30ab8040-39a2-48a1-8dbd-f543a30b379d-kube-api-access-j7mmw\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.059643 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059529    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-registry-tls\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.059643 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-trusted-ca\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.059643 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.059621    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30ab8040-39a2-48a1-8dbd-f543a30b379d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.061650 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.061628    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30ab8040-39a2-48a1-8dbd-f543a30b379d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.072224 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.072177    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7mmw\" (UniqueName: \"kubernetes.io/projected/30ab8040-39a2-48a1-8dbd-f543a30b379d-kube-api-access-j7mmw\") pod \"insights-runtime-extractor-8rkr6\" (UID: \"30ab8040-39a2-48a1-8dbd-f543a30b379d\") " pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.126541 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.126515    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8rkr6"
May 28 15:10:02.160425 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.160392    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-image-registry-private-configuration\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.160570 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.160429    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-bound-sa-token\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.160570 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.160449    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-registry-tls\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.160570 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.160465    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-trusted-ca\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.160570 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.160528    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-registry-certificates\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.160756 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.160594    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-installation-pull-secrets\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.160756 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.160663    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7qnb5\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-kube-api-access-7qnb5\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.160756 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.160696    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1c52025-a560-49f8-8531-dbe401ff5485-ca-trust-extracted\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.161087 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.161063    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1c52025-a560-49f8-8531-dbe401ff5485-ca-trust-extracted\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.161475 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.161428    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-trusted-ca\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.163506 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.163483    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-image-registry-private-configuration\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.163612 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.163485    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-registry-certificates\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.164206 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.164189    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-registry-tls\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.165234 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.165205    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-installation-pull-secrets\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.173857 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.173414    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.173857 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.173618    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qnb5\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-kube-api-access-7qnb5\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.174374 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.173951    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-bound-sa-token\") pod \"image-registry-94cdc59b5-h52qg\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") " pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.179171 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.179104    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:02.255786 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.255757    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8rkr6"]
May 28 15:10:02.258474 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:10:02.258448    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ab8040_39a2_48a1_8dbd_f543a30b379d.slice/crio-4596a6f093e17d9fb14dd2fd27a24410e4adf9c01b9c3f68452949c3d20e0c39 WatchSource:0}: Error finding container 4596a6f093e17d9fb14dd2fd27a24410e4adf9c01b9c3f68452949c3d20e0c39: Status 404 returned error can't find the container with id 4596a6f093e17d9fb14dd2fd27a24410e4adf9c01b9c3f68452949c3d20e0c39
May 28 15:10:02.260897 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.260875    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-installation-pull-secrets\") pod \"d1c52025-a560-49f8-8531-dbe401ff5485\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") "
May 28 15:10:02.260995 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.260921    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1c52025-a560-49f8-8531-dbe401ff5485-ca-trust-extracted\") pod \"d1c52025-a560-49f8-8531-dbe401ff5485\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") "
May 28 15:10:02.260995 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.260950    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-registry-tls\") pod \"d1c52025-a560-49f8-8531-dbe401ff5485\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") "
May 28 15:10:02.260995 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.260982    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-bound-sa-token\") pod \"d1c52025-a560-49f8-8531-dbe401ff5485\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") "
May 28 15:10:02.261151 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.261039    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-trusted-ca\") pod \"d1c52025-a560-49f8-8531-dbe401ff5485\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") "
May 28 15:10:02.261151 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.261074    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-registry-certificates\") pod \"d1c52025-a560-49f8-8531-dbe401ff5485\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") "
May 28 15:10:02.261151 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.261099    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qnb5\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-kube-api-access-7qnb5\") pod \"d1c52025-a560-49f8-8531-dbe401ff5485\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") "
May 28 15:10:02.261151 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.261127    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-image-registry-private-configuration\") pod \"d1c52025-a560-49f8-8531-dbe401ff5485\" (UID: \"d1c52025-a560-49f8-8531-dbe401ff5485\") "
May 28 15:10:02.261338 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.261156    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c52025-a560-49f8-8531-dbe401ff5485-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d1c52025-a560-49f8-8531-dbe401ff5485" (UID: "d1c52025-a560-49f8-8531-dbe401ff5485"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:10:02.261338 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.261300    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1c52025-a560-49f8-8531-dbe401ff5485-ca-trust-extracted\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:10:02.261544 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.261514    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d1c52025-a560-49f8-8531-dbe401ff5485" (UID: "d1c52025-a560-49f8-8531-dbe401ff5485"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 15:10:02.261658 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.261544    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d1c52025-a560-49f8-8531-dbe401ff5485" (UID: "d1c52025-a560-49f8-8531-dbe401ff5485"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 15:10:02.263413 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.263381    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d1c52025-a560-49f8-8531-dbe401ff5485" (UID: "d1c52025-a560-49f8-8531-dbe401ff5485"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 15:10:02.263531 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.263461    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-kube-api-access-7qnb5" (OuterVolumeSpecName: "kube-api-access-7qnb5") pod "d1c52025-a560-49f8-8531-dbe401ff5485" (UID: "d1c52025-a560-49f8-8531-dbe401ff5485"). InnerVolumeSpecName "kube-api-access-7qnb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:10:02.263531 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.263508    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d1c52025-a560-49f8-8531-dbe401ff5485" (UID: "d1c52025-a560-49f8-8531-dbe401ff5485"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:10:02.263801 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.263575    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d1c52025-a560-49f8-8531-dbe401ff5485" (UID: "d1c52025-a560-49f8-8531-dbe401ff5485"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:10:02.263801 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.263585    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d1c52025-a560-49f8-8531-dbe401ff5485" (UID: "d1c52025-a560-49f8-8531-dbe401ff5485"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 15:10:02.362652 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.362576    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-image-registry-private-configuration\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:10:02.362652 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.362611    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1c52025-a560-49f8-8531-dbe401ff5485-installation-pull-secrets\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:10:02.362652 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.362622    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-registry-tls\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:10:02.362652 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.362631    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-bound-sa-token\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:10:02.362652 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.362640    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-trusted-ca\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:10:02.362652 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.362649    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1c52025-a560-49f8-8531-dbe401ff5485-registry-certificates\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:10:02.362652 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:02.362657    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qnb5\" (UniqueName: \"kubernetes.io/projected/d1c52025-a560-49f8-8531-dbe401ff5485-kube-api-access-7qnb5\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:10:03.176844 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:03.176773    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94cdc59b5-h52qg"
May 28 15:10:03.176844 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:03.176800    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8rkr6" event={"ID":"30ab8040-39a2-48a1-8dbd-f543a30b379d","Type":"ContainerStarted","Data":"79d1f00f8331766e97b7d353fb61a39c4115f44c354e6d26eeb0ef24c8f47f43"}
May 28 15:10:03.176844 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:03.176834    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8rkr6" event={"ID":"30ab8040-39a2-48a1-8dbd-f543a30b379d","Type":"ContainerStarted","Data":"840024e0c05be60cc5b630a3d08dd6c53d5f19d0820bf65bc21697deee0d233d"}
May 28 15:10:03.176844 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:03.176843    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8rkr6" event={"ID":"30ab8040-39a2-48a1-8dbd-f543a30b379d","Type":"ContainerStarted","Data":"4596a6f093e17d9fb14dd2fd27a24410e4adf9c01b9c3f68452949c3d20e0c39"}
May 28 15:10:03.207049 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:03.207025    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-94cdc59b5-h52qg"]
May 28 15:10:03.217748 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:03.217726    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-94cdc59b5-h52qg"]
May 28 15:10:03.783082 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:03.783051    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1c52025-a560-49f8-8531-dbe401ff5485" path="/var/lib/kubelet/pods/d1c52025-a560-49f8-8531-dbe401ff5485/volumes"
May 28 15:10:05.183815 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:05.183784    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8rkr6" event={"ID":"30ab8040-39a2-48a1-8dbd-f543a30b379d","Type":"ContainerStarted","Data":"0eb99012f2d5fc328cd21e95858899f0c6f35b6294fcee04c1e343cc4f8d4eee"}
May 28 15:10:05.202685 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:05.202641    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8rkr6" podStartSLOduration=1.9773035079999999 podStartE2EDuration="4.202628213s" podCreationTimestamp="2026-05-28 15:10:01 +0000 UTC" firstStartedPulling="2026-05-28 15:10:02.308152576 +0000 UTC m=+139.093303214" lastFinishedPulling="2026-05-28 15:10:04.53347728 +0000 UTC m=+141.318627919" observedRunningTime="2026-05-28 15:10:05.200920306 +0000 UTC m=+141.986070966" watchObservedRunningTime="2026-05-28 15:10:05.202628213 +0000 UTC m=+141.987778873"
May 28 15:10:07.710322 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:07.710272    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" podUID="1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 15:10:13.932439 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.932399    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-575b5685c-8vtl5"]
May 28 15:10:13.934395 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.934378    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:13.936977 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.936948    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 28 15:10:13.937105 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.937066    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 28 15:10:13.937168 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.937113    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 28 15:10:13.937225 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.937205    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 28 15:10:13.937279 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.937228    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 28 15:10:13.938351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.938332    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 28 15:10:13.938475 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.938398    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wzvkg\""
May 28 15:10:13.938607 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.938594    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 28 15:10:13.943397 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.943374    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 28 15:10:13.944498 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:13.944472    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-575b5685c-8vtl5"]
May 28 15:10:14.050052 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.050020    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-oauth-config\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.050052 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.050049    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-oauth-serving-cert\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.050277 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.050128    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6wts\" (UniqueName: \"kubernetes.io/projected/75aaeaaa-42ea-4370-acc4-0fcf770e6638-kube-api-access-t6wts\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.050277 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.050168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-config\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.050277 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.050192    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-trusted-ca-bundle\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.050277 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.050214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-service-ca\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.050277 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.050234    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-serving-cert\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.151509 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.151462    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-oauth-config\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.151509 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.151509    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-oauth-serving-cert\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.151742 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.151597    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t6wts\" (UniqueName: \"kubernetes.io/projected/75aaeaaa-42ea-4370-acc4-0fcf770e6638-kube-api-access-t6wts\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.151742 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.151624    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-config\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.151742 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.151643    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-trusted-ca-bundle\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.151742 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.151662    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-service-ca\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.151742 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.151680    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-serving-cert\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.152351 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.152320    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-config\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.152665 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.152320    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-oauth-serving-cert\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.152665 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.152387    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-service-ca\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.152665 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.152623    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-trusted-ca-bundle\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.154013 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.153989    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-oauth-config\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.154212 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.154195    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-serving-cert\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.160363 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.160343    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6wts\" (UniqueName: \"kubernetes.io/projected/75aaeaaa-42ea-4370-acc4-0fcf770e6638-kube-api-access-t6wts\") pod \"console-575b5685c-8vtl5\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") " pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.245062 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.244995    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:14.368170 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:14.368141    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-575b5685c-8vtl5"]
May 28 15:10:14.371531 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:10:14.371510    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75aaeaaa_42ea_4370_acc4_0fcf770e6638.slice/crio-52f67f8026304c7e2dc35853293d63294dec4acdaf0d99c51497c6437256d539 WatchSource:0}: Error finding container 52f67f8026304c7e2dc35853293d63294dec4acdaf0d99c51497c6437256d539: Status 404 returned error can't find the container with id 52f67f8026304c7e2dc35853293d63294dec4acdaf0d99c51497c6437256d539
May 28 15:10:15.212053 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:15.209492    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-575b5685c-8vtl5" event={"ID":"75aaeaaa-42ea-4370-acc4-0fcf770e6638","Type":"ContainerStarted","Data":"52f67f8026304c7e2dc35853293d63294dec4acdaf0d99c51497c6437256d539"}
May 28 15:10:17.215943 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:17.215855    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-575b5685c-8vtl5" event={"ID":"75aaeaaa-42ea-4370-acc4-0fcf770e6638","Type":"ContainerStarted","Data":"626cdd8cf200645c80affe050cbd3735b76cd562f0165db335de9a76380be31e"}
May 28 15:10:17.233810 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:17.233764    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-575b5685c-8vtl5" podStartSLOduration=1.7009338870000001 podStartE2EDuration="4.233753118s" podCreationTimestamp="2026-05-28 15:10:13 +0000 UTC" firstStartedPulling="2026-05-28 15:10:14.373285308 +0000 UTC m=+151.158435947" lastFinishedPulling="2026-05-28 15:10:16.90610454 +0000 UTC m=+153.691255178" observedRunningTime="2026-05-28 15:10:17.232743071 +0000 UTC m=+154.017893731" watchObservedRunningTime="2026-05-28 15:10:17.233753118 +0000 UTC m=+154.018903778"
May 28 15:10:17.710671 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:17.710630    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" podUID="1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 15:10:22.136066 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:10:22.136031    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vs6bq" podUID="cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902"
May 28 15:10:22.148248 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:10:22.148211    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-nwwjb" podUID="ae300cc7-3fd0-4422-816d-eec9479c9a0b"
May 28 15:10:22.231006 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:22.228977    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vs6bq"
May 28 15:10:24.245831 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:24.245799    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:24.246192 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:24.245844    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:10:24.247045 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:24.247024    2560 patch_prober.go:28] interesting pod/console-575b5685c-8vtl5 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body=
May 28 15:10:24.247099 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:24.247077    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-575b5685c-8vtl5" podUID="75aaeaaa-42ea-4370-acc4-0fcf770e6638" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused"
May 28 15:10:26.379840 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.379805    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"]
May 28 15:10:26.382505 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.382488    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.385240 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.385219    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
May 28 15:10:26.385356 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.385223    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 15:10:26.385424 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.385382    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 15:10:26.386399 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.386379    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 15:10:26.386476 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.386386    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-n2rdq\""
May 28 15:10:26.386476 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.386414    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
May 28 15:10:26.395922 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.395895    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"]
May 28 15:10:26.401125 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.401091    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-grrmk"]
May 28 15:10:26.404424 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.404401    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.407065 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.407046    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qhxdm\""
May 28 15:10:26.407169 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.407094    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 15:10:26.407233 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.407195    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 15:10:26.407233 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.407195    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 15:10:26.544336 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544296    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srpll\" (UniqueName: \"kubernetes.io/projected/a059a4f0-1102-4713-a599-251fefeb0bd8-kube-api-access-srpll\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.544516 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc5wc\" (UniqueName: \"kubernetes.io/projected/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-kube-api-access-jc5wc\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.544516 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a059a4f0-1102-4713-a599-251fefeb0bd8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.544516 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544404    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.544516 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-accelerators-collector-config\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.544516 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544459    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a059a4f0-1102-4713-a599-251fefeb0bd8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.544723 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544521    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-textfile\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.544723 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-metrics-client-ca\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.544723 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544632    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a059a4f0-1102-4713-a599-251fefeb0bd8-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.544723 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544665    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-tls\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.544723 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544696    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-wtmp\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.544723 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544713    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-sys\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.544910 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.544739    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-root\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.645730 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645651    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-root\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.645730 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645694    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-srpll\" (UniqueName: \"kubernetes.io/projected/a059a4f0-1102-4713-a599-251fefeb0bd8-kube-api-access-srpll\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.645908 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645760    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-root\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.645908 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645803    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jc5wc\" (UniqueName: \"kubernetes.io/projected/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-kube-api-access-jc5wc\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.645908 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645831    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a059a4f0-1102-4713-a599-251fefeb0bd8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.645908 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645860    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.645908 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645888    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-accelerators-collector-config\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646132 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645918    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a059a4f0-1102-4713-a599-251fefeb0bd8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.646132 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645950    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-textfile\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646132 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.645973    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-metrics-client-ca\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646132 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646076    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a059a4f0-1102-4713-a599-251fefeb0bd8-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.646132 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646118    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-tls\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646432 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646149    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-wtmp\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646432 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646182    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-sys\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646432 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646270    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-sys\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646603 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646584    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-textfile\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646667 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646633    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-metrics-client-ca\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646727 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646708    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-wtmp\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.646818 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646787    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-accelerators-collector-config\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.647110 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.646860    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a059a4f0-1102-4713-a599-251fefeb0bd8-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.648421 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.648397    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a059a4f0-1102-4713-a599-251fefeb0bd8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.648633 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.648614    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.649112 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.649090    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-node-exporter-tls\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.649254 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.649230    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a059a4f0-1102-4713-a599-251fefeb0bd8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.653825 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.653796    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc5wc\" (UniqueName: \"kubernetes.io/projected/b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7-kube-api-access-jc5wc\") pod \"node-exporter-grrmk\" (UID: \"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7\") " pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.654064 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.654044    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srpll\" (UniqueName: \"kubernetes.io/projected/a059a4f0-1102-4713-a599-251fefeb0bd8-kube-api-access-srpll\") pod \"openshift-state-metrics-9c959559f-cv4tn\" (UID: \"a059a4f0-1102-4713-a599-251fefeb0bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.690954 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.690930    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"
May 28 15:10:26.712706 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.712675    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-grrmk"
May 28 15:10:26.723114 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:10:26.723085    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb929aa2f_d9d3_42e4_a0bd_44d6e5b90ae7.slice/crio-07f3be0e6331399504b8679a245be879e654a26cb42db50456d7ecb476aea67e WatchSource:0}: Error finding container 07f3be0e6331399504b8679a245be879e654a26cb42db50456d7ecb476aea67e: Status 404 returned error can't find the container with id 07f3be0e6331399504b8679a245be879e654a26cb42db50456d7ecb476aea67e
May 28 15:10:26.821320 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:26.821288    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn"]
May 28 15:10:26.826897 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:10:26.826870    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda059a4f0_1102_4713_a599_251fefeb0bd8.slice/crio-e5fedfbe64c5b70878a3ca2e7755bc4203a5cc204c4c3dad2dca887c92eac3ff WatchSource:0}: Error finding container e5fedfbe64c5b70878a3ca2e7755bc4203a5cc204c4c3dad2dca887c92eac3ff: Status 404 returned error can't find the container with id e5fedfbe64c5b70878a3ca2e7755bc4203a5cc204c4c3dad2dca887c92eac3ff
May 28 15:10:27.049257 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.049224    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:10:27.049425 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.049265    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:10:27.051739 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.051714    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902-metrics-tls\") pod \"dns-default-vs6bq\" (UID: \"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902\") " pod="openshift-dns/dns-default-vs6bq"
May 28 15:10:27.051841 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.051821    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae300cc7-3fd0-4422-816d-eec9479c9a0b-cert\") pod \"ingress-canary-nwwjb\" (UID: \"ae300cc7-3fd0-4422-816d-eec9479c9a0b\") " pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:10:27.241779 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.241688    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-grrmk" event={"ID":"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7","Type":"ContainerStarted","Data":"07f3be0e6331399504b8679a245be879e654a26cb42db50456d7ecb476aea67e"}
May 28 15:10:27.243325 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.243300    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn" event={"ID":"a059a4f0-1102-4713-a599-251fefeb0bd8","Type":"ContainerStarted","Data":"6f73e0214e59227cc6d835f857d83dfa35fb439d456b7eff71b53baeb797e049"}
May 28 15:10:27.243457 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.243331    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn" event={"ID":"a059a4f0-1102-4713-a599-251fefeb0bd8","Type":"ContainerStarted","Data":"f7d01d77df2928841f0eb428f4791df4a0586525340095157879263c95a566a9"}
May 28 15:10:27.243457 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.243341    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn" event={"ID":"a059a4f0-1102-4713-a599-251fefeb0bd8","Type":"ContainerStarted","Data":"e5fedfbe64c5b70878a3ca2e7755bc4203a5cc204c4c3dad2dca887c92eac3ff"}
May 28 15:10:27.332295 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.332270    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b5hqh\""
May 28 15:10:27.340360 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.340344    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vs6bq"
May 28 15:10:27.481344 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.481286    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vs6bq"]
May 28 15:10:27.485734 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:10:27.485699    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef8aaac_b3b6_4a7a_bc07_6c40d2f2b902.slice/crio-f068429fe1a6f12f97ae27e9969325ff78921fdeacf70be1b38af702a9947b5f WatchSource:0}: Error finding container f068429fe1a6f12f97ae27e9969325ff78921fdeacf70be1b38af702a9947b5f: Status 404 returned error can't find the container with id f068429fe1a6f12f97ae27e9969325ff78921fdeacf70be1b38af702a9947b5f
May 28 15:10:27.710525 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.710482    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" podUID="1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 15:10:27.710726 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.710594    2560 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg"
May 28 15:10:27.711189 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.711154    2560 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"be61873a960cd1fc17ed16cbecd03427d81f9b8720a160b70a75408d398f1cd7"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 28 15:10:27.711285 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:27.711213    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" podUID="1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95" containerName="service-proxy" containerID="cri-o://be61873a960cd1fc17ed16cbecd03427d81f9b8720a160b70a75408d398f1cd7" gracePeriod=30
May 28 15:10:28.248276 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:28.248244    2560 generic.go:358] "Generic (PLEG): container finished" podID="1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95" containerID="be61873a960cd1fc17ed16cbecd03427d81f9b8720a160b70a75408d398f1cd7" exitCode=2
May 28 15:10:28.248399 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:28.248370    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" event={"ID":"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95","Type":"ContainerDied","Data":"be61873a960cd1fc17ed16cbecd03427d81f9b8720a160b70a75408d398f1cd7"}
May 28 15:10:28.248475 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:28.248438    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9cf4d5fc9-jkdgg" event={"ID":"1b1f4b61-e0a7-4b31-b890-8e5c84e0ea95","Type":"ContainerStarted","Data":"c248010d7ad6ab41a9dea84a5040b812d6bcf9ffbeca19f129c4db51e1d3959b"}
May 28 15:10:28.250290 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:28.250198    2560 generic.go:358] "Generic (PLEG): container finished" podID="b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7" containerID="6d1b0962d8023f94046ac6ebc808e8ddbb8ec4b642a4d070fb516573eb815e2d" exitCode=0
May 28 15:10:28.250290 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:28.250231    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-grrmk" event={"ID":"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7","Type":"ContainerDied","Data":"6d1b0962d8023f94046ac6ebc808e8ddbb8ec4b642a4d070fb516573eb815e2d"}
May 28 15:10:28.253515 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:28.253494    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn" event={"ID":"a059a4f0-1102-4713-a599-251fefeb0bd8","Type":"ContainerStarted","Data":"3cd6488d143b5b148ba1c64ddd0563de25b75be7efb29bc8fa9f7ddf21ebcc18"}
May 28 15:10:28.254623 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:28.254583    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vs6bq" event={"ID":"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902","Type":"ContainerStarted","Data":"f068429fe1a6f12f97ae27e9969325ff78921fdeacf70be1b38af702a9947b5f"}
May 28 15:10:28.302826 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:28.302775    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-cv4tn" podStartSLOduration=1.069234858 podStartE2EDuration="2.302755605s" podCreationTimestamp="2026-05-28 15:10:26 +0000 UTC" firstStartedPulling="2026-05-28 15:10:26.940099989 +0000 UTC m=+163.725250628" lastFinishedPulling="2026-05-28 15:10:28.173620729 +0000 UTC m=+164.958771375" observedRunningTime="2026-05-28 15:10:28.302274091 +0000 UTC m=+165.087424751" watchObservedRunningTime="2026-05-28 15:10:28.302755605 +0000 UTC m=+165.087906265"
May 28 15:10:29.258847 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:29.258765    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-grrmk" event={"ID":"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7","Type":"ContainerStarted","Data":"17212279ce07310eb73be02d9e1d27492624ed825c341c17752ab8d4404201f9"}
May 28 15:10:29.258847 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:29.258806    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-grrmk" event={"ID":"b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7","Type":"ContainerStarted","Data":"3e57d1a199da439e0fa66ecc71cfd1754f9e4936d85657a88d9ecb84865e4744"}
May 28 15:10:29.260458 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:29.260432    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vs6bq" event={"ID":"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902","Type":"ContainerStarted","Data":"0c86a3de2ae7d90271f6f7282850bfe2b83411e443ed9bdff62e4b9ed77e0036"}
May 28 15:10:29.260458 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:29.260460    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vs6bq" event={"ID":"cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902","Type":"ContainerStarted","Data":"53ffc0e0fd18163451d1a3e449c12cf47554629eb4ce741614a3dc79f45028b2"}
May 28 15:10:29.261221 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:29.261202    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vs6bq"
May 28 15:10:29.281727 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:29.281680    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-grrmk" podStartSLOduration=2.229523857 podStartE2EDuration="3.281666811s" podCreationTimestamp="2026-05-28 15:10:26 +0000 UTC" firstStartedPulling="2026-05-28 15:10:26.725070694 +0000 UTC m=+163.510221333" lastFinishedPulling="2026-05-28 15:10:27.777213649 +0000 UTC m=+164.562364287" observedRunningTime="2026-05-28 15:10:29.280818018 +0000 UTC m=+166.065968679" watchObservedRunningTime="2026-05-28 15:10:29.281666811 +0000 UTC m=+166.066817539"
May 28 15:10:29.300878 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:29.300835    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-vs6bq" podStartSLOduration=128.930588356 podStartE2EDuration="2m10.300821524s" podCreationTimestamp="2026-05-28 15:08:19 +0000 UTC" firstStartedPulling="2026-05-28 15:10:27.488223789 +0000 UTC m=+164.273374432" lastFinishedPulling="2026-05-28 15:10:28.858456959 +0000 UTC m=+165.643607600" observedRunningTime="2026-05-28 15:10:29.300140849 +0000 UTC m=+166.085291506" watchObservedRunningTime="2026-05-28 15:10:29.300821524 +0000 UTC m=+166.085972183"
May 28 15:10:31.155992 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.155955    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"]
May 28 15:10:31.159036 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.159020    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"
May 28 15:10:31.162544 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.162520    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 28 15:10:31.162669 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.162635    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-cgr8b\""
May 28 15:10:31.168099 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.168076    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"]
May 28 15:10:31.281079 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.281041    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4e22aae7-e299-4b3e-b9ff-03811e253020-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-2pr9r\" (UID: \"4e22aae7-e299-4b3e-b9ff-03811e253020\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"
May 28 15:10:31.381927 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.381894    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4e22aae7-e299-4b3e-b9ff-03811e253020-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-2pr9r\" (UID: \"4e22aae7-e299-4b3e-b9ff-03811e253020\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"
May 28 15:10:31.384495 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.384470    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4e22aae7-e299-4b3e-b9ff-03811e253020-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-2pr9r\" (UID: \"4e22aae7-e299-4b3e-b9ff-03811e253020\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"
May 28 15:10:31.468500 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.468426    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"
May 28 15:10:31.589469 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:31.589438    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"]
May 28 15:10:31.592157 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:10:31.592132    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e22aae7_e299_4b3e_b9ff_03811e253020.slice/crio-5bfbf30ed6c0626684538fab7c8d5a57a15d2a506dcfa9db19d40062cc6f0c7d WatchSource:0}: Error finding container 5bfbf30ed6c0626684538fab7c8d5a57a15d2a506dcfa9db19d40062cc6f0c7d: Status 404 returned error can't find the container with id 5bfbf30ed6c0626684538fab7c8d5a57a15d2a506dcfa9db19d40062cc6f0c7d
May 28 15:10:32.270856 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:32.270818    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r" event={"ID":"4e22aae7-e299-4b3e-b9ff-03811e253020","Type":"ContainerStarted","Data":"5bfbf30ed6c0626684538fab7c8d5a57a15d2a506dcfa9db19d40062cc6f0c7d"}
May 28 15:10:33.274362 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:33.274279    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r" event={"ID":"4e22aae7-e299-4b3e-b9ff-03811e253020","Type":"ContainerStarted","Data":"9b91f127e88d667ce88e1c720b062bbc83d5450189a27e4c6ed168e0d67a6d8b"}
May 28 15:10:33.274750 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:33.274476    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"
May 28 15:10:33.279072 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:33.279044    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r"
May 28 15:10:33.289933 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:33.289891    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-2pr9r" podStartSLOduration=0.86178708 podStartE2EDuration="2.289879917s" podCreationTimestamp="2026-05-28 15:10:31 +0000 UTC" firstStartedPulling="2026-05-28 15:10:31.594123164 +0000 UTC m=+168.379273809" lastFinishedPulling="2026-05-28 15:10:33.022215987 +0000 UTC m=+169.807366646" observedRunningTime="2026-05-28 15:10:33.288657359 +0000 UTC m=+170.073808019" watchObservedRunningTime="2026-05-28 15:10:33.289879917 +0000 UTC m=+170.075030577"
May 28 15:10:34.246376 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:34.246310    2560 patch_prober.go:28] interesting pod/console-575b5685c-8vtl5 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body=
May 28 15:10:34.246598 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:34.246397    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-575b5685c-8vtl5" podUID="75aaeaaa-42ea-4370-acc4-0fcf770e6638" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused"
May 28 15:10:36.780746 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:36.780709    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:10:36.783277 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:36.783257    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n5mwm\""
May 28 15:10:36.791468 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:36.791450    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwwjb"
May 28 15:10:36.909675 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:36.909647    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwwjb"]
May 28 15:10:36.912655 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:10:36.912624    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae300cc7_3fd0_4422_816d_eec9479c9a0b.slice/crio-a5ca7afb9d2550c2701a4c6a49f2fcce67f3d4635248a97892f4a7b552500b70 WatchSource:0}: Error finding container a5ca7afb9d2550c2701a4c6a49f2fcce67f3d4635248a97892f4a7b552500b70: Status 404 returned error can't find the container with id a5ca7afb9d2550c2701a4c6a49f2fcce67f3d4635248a97892f4a7b552500b70
May 28 15:10:37.285520 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:37.285480    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwwjb" event={"ID":"ae300cc7-3fd0-4422-816d-eec9479c9a0b","Type":"ContainerStarted","Data":"a5ca7afb9d2550c2701a4c6a49f2fcce67f3d4635248a97892f4a7b552500b70"}
May 28 15:10:38.118664 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:38.118626    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-575b5685c-8vtl5"]
May 28 15:10:39.265979 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:39.265950    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vs6bq"
May 28 15:10:39.292433 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:39.292397    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwwjb" event={"ID":"ae300cc7-3fd0-4422-816d-eec9479c9a0b","Type":"ContainerStarted","Data":"b2ed39636c557aadb14c4062fb134d2ec733db6b690297f935fade673cd9fef3"}
May 28 15:10:39.308014 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:39.307966    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nwwjb" podStartSLOduration=138.721564538 podStartE2EDuration="2m20.307954271s" podCreationTimestamp="2026-05-28 15:08:19 +0000 UTC" firstStartedPulling="2026-05-28 15:10:36.914618602 +0000 UTC m=+173.699769241" lastFinishedPulling="2026-05-28 15:10:38.501008335 +0000 UTC m=+175.286158974" observedRunningTime="2026-05-28 15:10:39.306791156 +0000 UTC m=+176.091941816" watchObservedRunningTime="2026-05-28 15:10:39.307954271 +0000 UTC m=+176.093104931"
May 28 15:10:49.951405 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:49.951367    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vs6bq_cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902/dns/0.log"
May 28 15:10:49.957298 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:49.957276    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vs6bq_cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902/kube-rbac-proxy/0.log"
May 28 15:10:50.513641 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:10:50.513614    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wgnqk_8f283641-0611-4d23-b85e-28cc5090beb1/dns-node-resolver/0.log"
May 28 15:11:03.140064 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.140001    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-575b5685c-8vtl5" podUID="75aaeaaa-42ea-4370-acc4-0fcf770e6638" containerName="console" containerID="cri-o://626cdd8cf200645c80affe050cbd3735b76cd562f0165db335de9a76380be31e" gracePeriod=15
May 28 15:11:03.355912 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.355888    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-575b5685c-8vtl5_75aaeaaa-42ea-4370-acc4-0fcf770e6638/console/0.log"
May 28 15:11:03.356059 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.355928    2560 generic.go:358] "Generic (PLEG): container finished" podID="75aaeaaa-42ea-4370-acc4-0fcf770e6638" containerID="626cdd8cf200645c80affe050cbd3735b76cd562f0165db335de9a76380be31e" exitCode=2
May 28 15:11:03.356059 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.355977    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-575b5685c-8vtl5" event={"ID":"75aaeaaa-42ea-4370-acc4-0fcf770e6638","Type":"ContainerDied","Data":"626cdd8cf200645c80affe050cbd3735b76cd562f0165db335de9a76380be31e"}
May 28 15:11:03.385545 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.385524    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-575b5685c-8vtl5_75aaeaaa-42ea-4370-acc4-0fcf770e6638/console/0.log"
May 28 15:11:03.385688 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.385597    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:11:03.414810 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.414739    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-serving-cert\") pod \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") "
May 28 15:11:03.414810 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.414780    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-config\") pod \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") "
May 28 15:11:03.414983 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.414819    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-trusted-ca-bundle\") pod \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") "
May 28 15:11:03.414983 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.414874    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-oauth-serving-cert\") pod \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") "
May 28 15:11:03.414983 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.414903    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-service-ca\") pod \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") "
May 28 15:11:03.414983 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.414932    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-oauth-config\") pod \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") "
May 28 15:11:03.414983 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.414971    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6wts\" (UniqueName: \"kubernetes.io/projected/75aaeaaa-42ea-4370-acc4-0fcf770e6638-kube-api-access-t6wts\") pod \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\" (UID: \"75aaeaaa-42ea-4370-acc4-0fcf770e6638\") "
May 28 15:11:03.415232 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.415205    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-config" (OuterVolumeSpecName: "console-config") pod "75aaeaaa-42ea-4370-acc4-0fcf770e6638" (UID: "75aaeaaa-42ea-4370-acc4-0fcf770e6638"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 15:11:03.415612 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.415408    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "75aaeaaa-42ea-4370-acc4-0fcf770e6638" (UID: "75aaeaaa-42ea-4370-acc4-0fcf770e6638"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 15:11:03.415612 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.415480    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "75aaeaaa-42ea-4370-acc4-0fcf770e6638" (UID: "75aaeaaa-42ea-4370-acc4-0fcf770e6638"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 15:11:03.415776 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.415623    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-service-ca" (OuterVolumeSpecName: "service-ca") pod "75aaeaaa-42ea-4370-acc4-0fcf770e6638" (UID: "75aaeaaa-42ea-4370-acc4-0fcf770e6638"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 15:11:03.417301 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.417271    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "75aaeaaa-42ea-4370-acc4-0fcf770e6638" (UID: "75aaeaaa-42ea-4370-acc4-0fcf770e6638"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 15:11:03.417688 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.417665    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75aaeaaa-42ea-4370-acc4-0fcf770e6638-kube-api-access-t6wts" (OuterVolumeSpecName: "kube-api-access-t6wts") pod "75aaeaaa-42ea-4370-acc4-0fcf770e6638" (UID: "75aaeaaa-42ea-4370-acc4-0fcf770e6638"). InnerVolumeSpecName "kube-api-access-t6wts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:11:03.417918 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.417890    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "75aaeaaa-42ea-4370-acc4-0fcf770e6638" (UID: "75aaeaaa-42ea-4370-acc4-0fcf770e6638"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 15:11:03.515849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.515821    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-oauth-serving-cert\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:11:03.515849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.515848    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-service-ca\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:11:03.516032 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.515858    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-oauth-config\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:11:03.516032 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.515869    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6wts\" (UniqueName: \"kubernetes.io/projected/75aaeaaa-42ea-4370-acc4-0fcf770e6638-kube-api-access-t6wts\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:11:03.516032 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.515878    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-serving-cert\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:11:03.516032 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.515886    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-console-config\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:11:03.516032 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:03.515895    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75aaeaaa-42ea-4370-acc4-0fcf770e6638-trusted-ca-bundle\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:11:04.359954 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:04.359927    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-575b5685c-8vtl5_75aaeaaa-42ea-4370-acc4-0fcf770e6638/console/0.log"
May 28 15:11:04.360399 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:04.360010    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-575b5685c-8vtl5" event={"ID":"75aaeaaa-42ea-4370-acc4-0fcf770e6638","Type":"ContainerDied","Data":"52f67f8026304c7e2dc35853293d63294dec4acdaf0d99c51497c6437256d539"}
May 28 15:11:04.360399 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:04.360028    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-575b5685c-8vtl5"
May 28 15:11:04.360399 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:04.360043    2560 scope.go:117] "RemoveContainer" containerID="626cdd8cf200645c80affe050cbd3735b76cd562f0165db335de9a76380be31e"
May 28 15:11:04.378918 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:04.378897    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-575b5685c-8vtl5"]
May 28 15:11:04.382435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:04.382410    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-575b5685c-8vtl5"]
May 28 15:11:05.789050 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:11:05.789018    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75aaeaaa-42ea-4370-acc4-0fcf770e6638" path="/var/lib/kubelet/pods/75aaeaaa-42ea-4370-acc4-0fcf770e6638/volumes"
May 28 15:12:24.420778 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.420747    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-sdbjv"]
May 28 15:12:24.421182 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.420990    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75aaeaaa-42ea-4370-acc4-0fcf770e6638" containerName="console"
May 28 15:12:24.421182 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.421003    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="75aaeaaa-42ea-4370-acc4-0fcf770e6638" containerName="console"
May 28 15:12:24.421182 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.421048    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="75aaeaaa-42ea-4370-acc4-0fcf770e6638" containerName="console"
May 28 15:12:24.422657 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.422639    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.425327 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.425302    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 15:12:24.432672 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.432650    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sdbjv"]
May 28 15:12:24.474127 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.474102    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f37780de-3086-4fb5-898e-ed20851982b7-original-pull-secret\") pod \"global-pull-secret-syncer-sdbjv\" (UID: \"f37780de-3086-4fb5-898e-ed20851982b7\") " pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.474290 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.474149    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f37780de-3086-4fb5-898e-ed20851982b7-dbus\") pod \"global-pull-secret-syncer-sdbjv\" (UID: \"f37780de-3086-4fb5-898e-ed20851982b7\") " pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.474290 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.474183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f37780de-3086-4fb5-898e-ed20851982b7-kubelet-config\") pod \"global-pull-secret-syncer-sdbjv\" (UID: \"f37780de-3086-4fb5-898e-ed20851982b7\") " pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.575002 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.574968    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f37780de-3086-4fb5-898e-ed20851982b7-original-pull-secret\") pod \"global-pull-secret-syncer-sdbjv\" (UID: \"f37780de-3086-4fb5-898e-ed20851982b7\") " pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.575165 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.575021    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f37780de-3086-4fb5-898e-ed20851982b7-dbus\") pod \"global-pull-secret-syncer-sdbjv\" (UID: \"f37780de-3086-4fb5-898e-ed20851982b7\") " pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.575165 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.575067    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f37780de-3086-4fb5-898e-ed20851982b7-kubelet-config\") pod \"global-pull-secret-syncer-sdbjv\" (UID: \"f37780de-3086-4fb5-898e-ed20851982b7\") " pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.575165 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.575142    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f37780de-3086-4fb5-898e-ed20851982b7-kubelet-config\") pod \"global-pull-secret-syncer-sdbjv\" (UID: \"f37780de-3086-4fb5-898e-ed20851982b7\") " pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.575333 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.575232    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f37780de-3086-4fb5-898e-ed20851982b7-dbus\") pod \"global-pull-secret-syncer-sdbjv\" (UID: \"f37780de-3086-4fb5-898e-ed20851982b7\") " pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.577413 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.577387    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f37780de-3086-4fb5-898e-ed20851982b7-original-pull-secret\") pod \"global-pull-secret-syncer-sdbjv\" (UID: \"f37780de-3086-4fb5-898e-ed20851982b7\") " pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.730647 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.730576    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdbjv"
May 28 15:12:24.850183 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:24.850044    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sdbjv"]
May 28 15:12:24.852848 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:12:24.852823    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf37780de_3086_4fb5_898e_ed20851982b7.slice/crio-b14462bc9096b642aecb0a64aa960e62f1837a831db4b435c1d064d4f436f96a WatchSource:0}: Error finding container b14462bc9096b642aecb0a64aa960e62f1837a831db4b435c1d064d4f436f96a: Status 404 returned error can't find the container with id b14462bc9096b642aecb0a64aa960e62f1837a831db4b435c1d064d4f436f96a
May 28 15:12:25.569726 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:25.569686    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sdbjv" event={"ID":"f37780de-3086-4fb5-898e-ed20851982b7","Type":"ContainerStarted","Data":"b14462bc9096b642aecb0a64aa960e62f1837a831db4b435c1d064d4f436f96a"}
May 28 15:12:29.582538 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:29.582498    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sdbjv" event={"ID":"f37780de-3086-4fb5-898e-ed20851982b7","Type":"ContainerStarted","Data":"8d097c86b18bcfeec729a968a70ce16bcbc4a2bfb0189f8b196290e9dc51d27f"}
May 28 15:12:29.597154 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:29.597108    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sdbjv" podStartSLOduration=1.479094326 podStartE2EDuration="5.597095048s" podCreationTimestamp="2026-05-28 15:12:24 +0000 UTC" firstStartedPulling="2026-05-28 15:12:24.854359066 +0000 UTC m=+281.639509704" lastFinishedPulling="2026-05-28 15:12:28.972359788 +0000 UTC m=+285.757510426" observedRunningTime="2026-05-28 15:12:29.596237562 +0000 UTC m=+286.381388223" watchObservedRunningTime="2026-05-28 15:12:29.597095048 +0000 UTC m=+286.382245733"
May 28 15:12:43.700266 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:12:43.700238    2560 kubelet.go:1628] "Image garbage collection succeeded"
May 28 15:13:26.183567 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.183517    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"]
May 28 15:13:26.185472 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.185449    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"
May 28 15:13:26.188140 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.188117    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
May 28 15:13:26.188280 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.188172    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-kml5s\""
May 28 15:13:26.188280 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.188239    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
May 28 15:13:26.198967 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.198946    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"]
May 28 15:13:26.294659 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.294623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcjm6\" (UniqueName: \"kubernetes.io/projected/39185aa1-79cb-410a-b0e7-586f36079917-kube-api-access-qcjm6\") pod \"cert-manager-operator-controller-manager-54b9655956-nt66g\" (UID: \"39185aa1-79cb-410a-b0e7-586f36079917\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"
May 28 15:13:26.294822 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.294671    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39185aa1-79cb-410a-b0e7-586f36079917-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-nt66g\" (UID: \"39185aa1-79cb-410a-b0e7-586f36079917\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"
May 28 15:13:26.395930 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.395893    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjm6\" (UniqueName: \"kubernetes.io/projected/39185aa1-79cb-410a-b0e7-586f36079917-kube-api-access-qcjm6\") pod \"cert-manager-operator-controller-manager-54b9655956-nt66g\" (UID: \"39185aa1-79cb-410a-b0e7-586f36079917\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"
May 28 15:13:26.395930 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.395938    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39185aa1-79cb-410a-b0e7-586f36079917-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-nt66g\" (UID: \"39185aa1-79cb-410a-b0e7-586f36079917\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"
May 28 15:13:26.396238 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.396223    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39185aa1-79cb-410a-b0e7-586f36079917-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-nt66g\" (UID: \"39185aa1-79cb-410a-b0e7-586f36079917\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"
May 28 15:13:26.403610 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.403586    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcjm6\" (UniqueName: \"kubernetes.io/projected/39185aa1-79cb-410a-b0e7-586f36079917-kube-api-access-qcjm6\") pod \"cert-manager-operator-controller-manager-54b9655956-nt66g\" (UID: \"39185aa1-79cb-410a-b0e7-586f36079917\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"
May 28 15:13:26.494636 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.494566    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"
May 28 15:13:26.630197 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.627822    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g"]
May 28 15:13:26.634619 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:13:26.634593    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39185aa1_79cb_410a_b0e7_586f36079917.slice/crio-15d1126dd575ec19500f4bcaf3d411a0baa7af1336a9e1abb0671fef7671a9c0 WatchSource:0}: Error finding container 15d1126dd575ec19500f4bcaf3d411a0baa7af1336a9e1abb0671fef7671a9c0: Status 404 returned error can't find the container with id 15d1126dd575ec19500f4bcaf3d411a0baa7af1336a9e1abb0671fef7671a9c0
May 28 15:13:26.636689 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.636674    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 15:13:26.725008 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:26.724980    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g" event={"ID":"39185aa1-79cb-410a-b0e7-586f36079917","Type":"ContainerStarted","Data":"15d1126dd575ec19500f4bcaf3d411a0baa7af1336a9e1abb0671fef7671a9c0"}
May 28 15:13:28.732152 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:28.732058    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g" event={"ID":"39185aa1-79cb-410a-b0e7-586f36079917","Type":"ContainerStarted","Data":"fa642280b531789d61dfaede8d35f8c7b1a793b3bce043a1b8a48852f9b7d1e9"}
May 28 15:13:28.755056 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:28.754994    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-nt66g" podStartSLOduration=0.989547423 podStartE2EDuration="2.754975824s" podCreationTimestamp="2026-05-28 15:13:26 +0000 UTC" firstStartedPulling="2026-05-28 15:13:26.636795902 +0000 UTC m=+343.421946541" lastFinishedPulling="2026-05-28 15:13:28.402224301 +0000 UTC m=+345.187374942" observedRunningTime="2026-05-28 15:13:28.752436283 +0000 UTC m=+345.537586937" watchObservedRunningTime="2026-05-28 15:13:28.754975824 +0000 UTC m=+345.540126491"
May 28 15:13:30.509442 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.509401    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"]
May 28 15:13:30.511906 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.511889    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.514337 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.514312    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 28 15:13:30.515591 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.515571    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 28 15:13:30.515591 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.515582    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ldgjz\""
May 28 15:13:30.521436 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.521411    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"]
May 28 15:13:30.625417 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.625378    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttz7w\" (UniqueName: \"kubernetes.io/projected/452d8661-09ab-46da-b092-2eb27c60c865-kube-api-access-ttz7w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.625417 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.625423    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.625689 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.625491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.725863 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.725823    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ttz7w\" (UniqueName: \"kubernetes.io/projected/452d8661-09ab-46da-b092-2eb27c60c865-kube-api-access-ttz7w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.726024 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.725879    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.726024 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.725918    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.726279 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.726256    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.726360 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.726294    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.734726 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.734704    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttz7w\" (UniqueName: \"kubernetes.io/projected/452d8661-09ab-46da-b092-2eb27c60c865-kube-api-access-ttz7w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.820725 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.820694    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:30.946495 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:30.946454    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"]
May 28 15:13:30.950383 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:13:30.950355    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452d8661_09ab_46da_b092_2eb27c60c865.slice/crio-2e9aece8201b6939e9d74fa6a441127dc9881fa34555a99b5230c38d9dc7ac30 WatchSource:0}: Error finding container 2e9aece8201b6939e9d74fa6a441127dc9881fa34555a99b5230c38d9dc7ac30: Status 404 returned error can't find the container with id 2e9aece8201b6939e9d74fa6a441127dc9881fa34555a99b5230c38d9dc7ac30
May 28 15:13:31.742252 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:31.742212    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn" event={"ID":"452d8661-09ab-46da-b092-2eb27c60c865","Type":"ContainerStarted","Data":"2e9aece8201b6939e9d74fa6a441127dc9881fa34555a99b5230c38d9dc7ac30"}
May 28 15:13:34.434073 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.434031    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-5spzh"]
May 28 15:13:34.436507 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.436476    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh"
May 28 15:13:34.439107 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.439085    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
May 28 15:13:34.440291 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.440266    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-nx7rl\""
May 28 15:13:34.440291 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.440266    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
May 28 15:13:34.446109 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.446085    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-5spzh"]
May 28 15:13:34.552897 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.552837    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbx5s\" (UniqueName: \"kubernetes.io/projected/65380962-dd5a-4ead-ac51-9717096c67b7-kube-api-access-bbx5s\") pod \"cert-manager-cainjector-68b757865b-5spzh\" (UID: \"65380962-dd5a-4ead-ac51-9717096c67b7\") " pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh"
May 28 15:13:34.553084 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.552906    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65380962-dd5a-4ead-ac51-9717096c67b7-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-5spzh\" (UID: \"65380962-dd5a-4ead-ac51-9717096c67b7\") " pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh"
May 28 15:13:34.654185 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.654138    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bbx5s\" (UniqueName: \"kubernetes.io/projected/65380962-dd5a-4ead-ac51-9717096c67b7-kube-api-access-bbx5s\") pod \"cert-manager-cainjector-68b757865b-5spzh\" (UID: \"65380962-dd5a-4ead-ac51-9717096c67b7\") " pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh"
May 28 15:13:34.654346 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.654209    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65380962-dd5a-4ead-ac51-9717096c67b7-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-5spzh\" (UID: \"65380962-dd5a-4ead-ac51-9717096c67b7\") " pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh"
May 28 15:13:34.663691 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.663658    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65380962-dd5a-4ead-ac51-9717096c67b7-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-5spzh\" (UID: \"65380962-dd5a-4ead-ac51-9717096c67b7\") " pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh"
May 28 15:13:34.666502 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.666480    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbx5s\" (UniqueName: \"kubernetes.io/projected/65380962-dd5a-4ead-ac51-9717096c67b7-kube-api-access-bbx5s\") pod \"cert-manager-cainjector-68b757865b-5spzh\" (UID: \"65380962-dd5a-4ead-ac51-9717096c67b7\") " pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh"
May 28 15:13:34.749063 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.748977    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh"
May 28 15:13:34.885042 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:34.884997    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-5spzh"]
May 28 15:13:34.886954 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:13:34.886926    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65380962_dd5a_4ead_ac51_9717096c67b7.slice/crio-2e89f19b3b32b3c1c706dad11942d52f9b11755eab982937b672246d7f11d28d WatchSource:0}: Error finding container 2e89f19b3b32b3c1c706dad11942d52f9b11755eab982937b672246d7f11d28d: Status 404 returned error can't find the container with id 2e89f19b3b32b3c1c706dad11942d52f9b11755eab982937b672246d7f11d28d
May 28 15:13:35.755170 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:35.755118    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh" event={"ID":"65380962-dd5a-4ead-ac51-9717096c67b7","Type":"ContainerStarted","Data":"2e89f19b3b32b3c1c706dad11942d52f9b11755eab982937b672246d7f11d28d"}
May 28 15:13:39.769046 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:39.768982    2560 generic.go:358] "Generic (PLEG): container finished" podID="452d8661-09ab-46da-b092-2eb27c60c865" containerID="4ee39d0867cae9afb51ab637fa190e5072f05a034fe5edba9b41a2ba7851fab1" exitCode=0
May 28 15:13:39.769541 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:39.769133    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn" event={"ID":"452d8661-09ab-46da-b092-2eb27c60c865","Type":"ContainerDied","Data":"4ee39d0867cae9afb51ab637fa190e5072f05a034fe5edba9b41a2ba7851fab1"}
May 28 15:13:39.770665 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:39.770605    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh" event={"ID":"65380962-dd5a-4ead-ac51-9717096c67b7","Type":"ContainerStarted","Data":"b038412397a37bbb19bfe02a530c403eb4468df621a724c44b7bc5b8f6bb433e"}
May 28 15:13:39.803854 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:39.803793    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-5spzh" podStartSLOduration=1.15402999 podStartE2EDuration="5.803772903s" podCreationTimestamp="2026-05-28 15:13:34 +0000 UTC" firstStartedPulling="2026-05-28 15:13:34.888702365 +0000 UTC m=+351.673853003" lastFinishedPulling="2026-05-28 15:13:39.538445268 +0000 UTC m=+356.323595916" observedRunningTime="2026-05-28 15:13:39.801488797 +0000 UTC m=+356.586639457" watchObservedRunningTime="2026-05-28 15:13:39.803772903 +0000 UTC m=+356.588923569"
May 28 15:13:42.780816 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:42.780733    2560 generic.go:358] "Generic (PLEG): container finished" podID="452d8661-09ab-46da-b092-2eb27c60c865" containerID="41a23b495b40bbee7234275c2140c5959a7d7026cf21480e53d20ea17ed5bf70" exitCode=0
May 28 15:13:42.780816 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:42.780798    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn" event={"ID":"452d8661-09ab-46da-b092-2eb27c60c865","Type":"ContainerDied","Data":"41a23b495b40bbee7234275c2140c5959a7d7026cf21480e53d20ea17ed5bf70"}
May 28 15:13:48.803144 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:48.803110    2560 generic.go:358] "Generic (PLEG): container finished" podID="452d8661-09ab-46da-b092-2eb27c60c865" containerID="442c8b8619356e95414ba3b43655c890bf9f6038f245d30ad5f9bce43652d532" exitCode=0
May 28 15:13:48.803503 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:48.803180    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn" event={"ID":"452d8661-09ab-46da-b092-2eb27c60c865","Type":"ContainerDied","Data":"442c8b8619356e95414ba3b43655c890bf9f6038f245d30ad5f9bce43652d532"}
May 28 15:13:49.933850 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:49.933827    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:13:50.073815 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.073730    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-util\") pod \"452d8661-09ab-46da-b092-2eb27c60c865\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") "
May 28 15:13:50.073815 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.073790    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttz7w\" (UniqueName: \"kubernetes.io/projected/452d8661-09ab-46da-b092-2eb27c60c865-kube-api-access-ttz7w\") pod \"452d8661-09ab-46da-b092-2eb27c60c865\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") "
May 28 15:13:50.073815 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.073814    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-bundle\") pod \"452d8661-09ab-46da-b092-2eb27c60c865\" (UID: \"452d8661-09ab-46da-b092-2eb27c60c865\") "
May 28 15:13:50.074225 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.074199    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-bundle" (OuterVolumeSpecName: "bundle") pod "452d8661-09ab-46da-b092-2eb27c60c865" (UID: "452d8661-09ab-46da-b092-2eb27c60c865"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:13:50.076044 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.076022    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452d8661-09ab-46da-b092-2eb27c60c865-kube-api-access-ttz7w" (OuterVolumeSpecName: "kube-api-access-ttz7w") pod "452d8661-09ab-46da-b092-2eb27c60c865" (UID: "452d8661-09ab-46da-b092-2eb27c60c865"). InnerVolumeSpecName "kube-api-access-ttz7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:13:50.079135 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.079107    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-util" (OuterVolumeSpecName: "util") pod "452d8661-09ab-46da-b092-2eb27c60c865" (UID: "452d8661-09ab-46da-b092-2eb27c60c865"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:13:50.174820 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.174780    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-util\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:13:50.174820 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.174814    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttz7w\" (UniqueName: \"kubernetes.io/projected/452d8661-09ab-46da-b092-2eb27c60c865-kube-api-access-ttz7w\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:13:50.174820 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.174825    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/452d8661-09ab-46da-b092-2eb27c60c865-bundle\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:13:50.809456 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.809423    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn" event={"ID":"452d8661-09ab-46da-b092-2eb27c60c865","Type":"ContainerDied","Data":"2e9aece8201b6939e9d74fa6a441127dc9881fa34555a99b5230c38d9dc7ac30"}
May 28 15:13:50.809456 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.809457    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e9aece8201b6939e9d74fa6a441127dc9881fa34555a99b5230c38d9dc7ac30"
May 28 15:13:50.809456 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:13:50.809459    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5sbfn"
May 28 15:14:19.914345 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.914306    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"]
May 28 15:14:19.914827 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.914542    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452d8661-09ab-46da-b092-2eb27c60c865" containerName="util"
May 28 15:14:19.914827 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.914566    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="452d8661-09ab-46da-b092-2eb27c60c865" containerName="util"
May 28 15:14:19.914827 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.914575    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452d8661-09ab-46da-b092-2eb27c60c865" containerName="pull"
May 28 15:14:19.914827 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.914581    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="452d8661-09ab-46da-b092-2eb27c60c865" containerName="pull"
May 28 15:14:19.914827 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.914592    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452d8661-09ab-46da-b092-2eb27c60c865" containerName="extract"
May 28 15:14:19.914827 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.914597    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="452d8661-09ab-46da-b092-2eb27c60c865" containerName="extract"
May 28 15:14:19.914827 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.914640    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="452d8661-09ab-46da-b092-2eb27c60c865" containerName="extract"
May 28 15:14:19.916427 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.916408    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:19.919235 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.919213    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 28 15:14:19.920573 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.920543    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ldgjz\""
May 28 15:14:19.920683 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.920585    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 28 15:14:19.929519 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.929494    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"]
May 28 15:14:19.992158 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.992133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:19.992277 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.992178    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:19.992277 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:19.992206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cjlz\" (UniqueName: \"kubernetes.io/projected/517d0df9-2d98-4512-ae07-ecff52d7e621-kube-api-access-4cjlz\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:20.092868 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.092836    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:20.092984 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.092883    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:20.092984 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.092907    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4cjlz\" (UniqueName: \"kubernetes.io/projected/517d0df9-2d98-4512-ae07-ecff52d7e621-kube-api-access-4cjlz\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:20.093179 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.093161    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:20.093220 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.093202    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:20.120582 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.120547    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cjlz\" (UniqueName: \"kubernetes.io/projected/517d0df9-2d98-4512-ae07-ecff52d7e621-kube-api-access-4cjlz\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:20.225403 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.225348    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:20.353390 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.353357    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"]
May 28 15:14:20.356068 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:14:20.356039    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517d0df9_2d98_4512_ae07_ecff52d7e621.slice/crio-b0d586e8600674393c88822dde11a8abf3c2685fdf8e18ee30a3a49deb914711 WatchSource:0}: Error finding container b0d586e8600674393c88822dde11a8abf3c2685fdf8e18ee30a3a49deb914711: Status 404 returned error can't find the container with id b0d586e8600674393c88822dde11a8abf3c2685fdf8e18ee30a3a49deb914711
May 28 15:14:20.883518 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.883482    2560 generic.go:358] "Generic (PLEG): container finished" podID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerID="7b248b3bd85bba9202c0f935968815981f16cc6cfac75e28219cf17eeaf585ea" exitCode=0
May 28 15:14:20.883732 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.883584    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97" event={"ID":"517d0df9-2d98-4512-ae07-ecff52d7e621","Type":"ContainerDied","Data":"7b248b3bd85bba9202c0f935968815981f16cc6cfac75e28219cf17eeaf585ea"}
May 28 15:14:20.883732 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:20.883618    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97" event={"ID":"517d0df9-2d98-4512-ae07-ecff52d7e621","Type":"ContainerStarted","Data":"b0d586e8600674393c88822dde11a8abf3c2685fdf8e18ee30a3a49deb914711"}
May 28 15:14:21.823255 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.823218    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"]
May 28 15:14:21.825232 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.825204    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:21.827934 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.827913    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 28 15:14:21.828072 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.828001    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c8lf9\""
May 28 15:14:21.828176 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.828159    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 28 15:14:21.828247 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.828188    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 28 15:14:21.828304 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.828283    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 28 15:14:21.840012 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.839987    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"]
May 28 15:14:21.888170 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.888146    2560 generic.go:358] "Generic (PLEG): container finished" podID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerID="2cf815c69d8c1d2fbb3bbdd0b292b29197934199ed4ab4fcb009df1821bb5e89" exitCode=0
May 28 15:14:21.888302 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.888180    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97" event={"ID":"517d0df9-2d98-4512-ae07-ecff52d7e621","Type":"ContainerDied","Data":"2cf815c69d8c1d2fbb3bbdd0b292b29197934199ed4ab4fcb009df1821bb5e89"}
May 28 15:14:21.910217 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.909001    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a0362333-2da4-44c2-ac26-30ffd0564bf3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-gdmjw\" (UID: \"a0362333-2da4-44c2-ac26-30ffd0564bf3\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:21.910217 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.909090    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm7n8\" (UniqueName: \"kubernetes.io/projected/a0362333-2da4-44c2-ac26-30ffd0564bf3-kube-api-access-sm7n8\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-gdmjw\" (UID: \"a0362333-2da4-44c2-ac26-30ffd0564bf3\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:21.910217 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:21.909187    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0362333-2da4-44c2-ac26-30ffd0564bf3-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-gdmjw\" (UID: \"a0362333-2da4-44c2-ac26-30ffd0564bf3\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:22.010488 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.010442    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a0362333-2da4-44c2-ac26-30ffd0564bf3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-gdmjw\" (UID: \"a0362333-2da4-44c2-ac26-30ffd0564bf3\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:22.010704 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.010514    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sm7n8\" (UniqueName: \"kubernetes.io/projected/a0362333-2da4-44c2-ac26-30ffd0564bf3-kube-api-access-sm7n8\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-gdmjw\" (UID: \"a0362333-2da4-44c2-ac26-30ffd0564bf3\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:22.010704 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.010597    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0362333-2da4-44c2-ac26-30ffd0564bf3-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-gdmjw\" (UID: \"a0362333-2da4-44c2-ac26-30ffd0564bf3\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:22.013345 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.013316    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0362333-2da4-44c2-ac26-30ffd0564bf3-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-gdmjw\" (UID: \"a0362333-2da4-44c2-ac26-30ffd0564bf3\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:22.013453 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.013382    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a0362333-2da4-44c2-ac26-30ffd0564bf3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-gdmjw\" (UID: \"a0362333-2da4-44c2-ac26-30ffd0564bf3\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:22.022150 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.022128    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm7n8\" (UniqueName: \"kubernetes.io/projected/a0362333-2da4-44c2-ac26-30ffd0564bf3-kube-api-access-sm7n8\") pod \"opendatahub-operator-controller-manager-57c44ff6d8-gdmjw\" (UID: \"a0362333-2da4-44c2-ac26-30ffd0564bf3\") " pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:22.134810 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.134781    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:22.276930 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.276907    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"]
May 28 15:14:22.279239 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:14:22.279208    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0362333_2da4_44c2_ac26_30ffd0564bf3.slice/crio-ac9435f717d54aba13ad345e26ad2002308d795a98e6b8b01af944500d55f37e WatchSource:0}: Error finding container ac9435f717d54aba13ad345e26ad2002308d795a98e6b8b01af944500d55f37e: Status 404 returned error can't find the container with id ac9435f717d54aba13ad345e26ad2002308d795a98e6b8b01af944500d55f37e
May 28 15:14:22.891960 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.891927    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw" event={"ID":"a0362333-2da4-44c2-ac26-30ffd0564bf3","Type":"ContainerStarted","Data":"ac9435f717d54aba13ad345e26ad2002308d795a98e6b8b01af944500d55f37e"}
May 28 15:14:22.893752 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.893720    2560 generic.go:358] "Generic (PLEG): container finished" podID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerID="581a55a53258ccdfd833844e5e84f7120f6ea6953ff419534e1f7c3a631eb838" exitCode=0
May 28 15:14:22.893892 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:22.893764    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97" event={"ID":"517d0df9-2d98-4512-ae07-ecff52d7e621","Type":"ContainerDied","Data":"581a55a53258ccdfd833844e5e84f7120f6ea6953ff419534e1f7c3a631eb838"}
May 28 15:14:24.761582 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.761539    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:24.832437 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.832414    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-util\") pod \"517d0df9-2d98-4512-ae07-ecff52d7e621\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") "
May 28 15:14:24.832522 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.832447    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-bundle\") pod \"517d0df9-2d98-4512-ae07-ecff52d7e621\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") "
May 28 15:14:24.832522 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.832465    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cjlz\" (UniqueName: \"kubernetes.io/projected/517d0df9-2d98-4512-ae07-ecff52d7e621-kube-api-access-4cjlz\") pod \"517d0df9-2d98-4512-ae07-ecff52d7e621\" (UID: \"517d0df9-2d98-4512-ae07-ecff52d7e621\") "
May 28 15:14:24.833283 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.833250    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-bundle" (OuterVolumeSpecName: "bundle") pod "517d0df9-2d98-4512-ae07-ecff52d7e621" (UID: "517d0df9-2d98-4512-ae07-ecff52d7e621"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:14:24.834608 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.834586    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/517d0df9-2d98-4512-ae07-ecff52d7e621-kube-api-access-4cjlz" (OuterVolumeSpecName: "kube-api-access-4cjlz") pod "517d0df9-2d98-4512-ae07-ecff52d7e621" (UID: "517d0df9-2d98-4512-ae07-ecff52d7e621"). InnerVolumeSpecName "kube-api-access-4cjlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:14:24.837797 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.837773    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-util" (OuterVolumeSpecName: "util") pod "517d0df9-2d98-4512-ae07-ecff52d7e621" (UID: "517d0df9-2d98-4512-ae07-ecff52d7e621"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:14:24.901767 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.901730    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw" event={"ID":"a0362333-2da4-44c2-ac26-30ffd0564bf3","Type":"ContainerStarted","Data":"8dd65fe57cfc34905768cd1544749737b281f2eb829cf68659e824cb0a7baa0d"}
May 28 15:14:24.901920 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.901802    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:24.903464 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.903440    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97" event={"ID":"517d0df9-2d98-4512-ae07-ecff52d7e621","Type":"ContainerDied","Data":"b0d586e8600674393c88822dde11a8abf3c2685fdf8e18ee30a3a49deb914711"}
May 28 15:14:24.903587 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.903468    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0d586e8600674393c88822dde11a8abf3c2685fdf8e18ee30a3a49deb914711"
May 28 15:14:24.903587 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.903496    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c95lg97"
May 28 15:14:24.933886 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.933863    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-util\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:14:24.933886 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.933883    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/517d0df9-2d98-4512-ae07-ecff52d7e621-bundle\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:14:24.934031 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:24.933974    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cjlz\" (UniqueName: \"kubernetes.io/projected/517d0df9-2d98-4512-ae07-ecff52d7e621-kube-api-access-4cjlz\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:14:25.830200 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:25.830147    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw" podStartSLOduration=2.306086121 podStartE2EDuration="4.830129419s" podCreationTimestamp="2026-05-28 15:14:21 +0000 UTC" firstStartedPulling="2026-05-28 15:14:22.280895699 +0000 UTC m=+399.066046339" lastFinishedPulling="2026-05-28 15:14:24.804938996 +0000 UTC m=+401.590089637" observedRunningTime="2026-05-28 15:14:24.950706836 +0000 UTC m=+401.735857496" watchObservedRunningTime="2026-05-28 15:14:25.830129419 +0000 UTC m=+402.615280128"
May 28 15:14:35.908933 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:35.908901    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57c44ff6d8-gdmjw"
May 28 15:14:41.154799 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.154761    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"]
May 28 15:14:41.155267 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.155115    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerName="pull"
May 28 15:14:41.155267 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.155131    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerName="pull"
May 28 15:14:41.155267 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.155152    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerName="extract"
May 28 15:14:41.155267 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.155160    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerName="extract"
May 28 15:14:41.155267 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.155178    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerName="util"
May 28 15:14:41.155267 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.155187    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerName="util"
May 28 15:14:41.155267 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.155247    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="517d0df9-2d98-4512-ae07-ecff52d7e621" containerName="extract"
May 28 15:14:41.158422 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.158398    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.161166 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.161144    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 28 15:14:41.161277 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.161167    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ldgjz\""
May 28 15:14:41.162119 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.162100    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 28 15:14:41.165637 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.165615    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"]
May 28 15:14:41.245976 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.245944    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.246122 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.245979    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.246122 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.245997    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgg72\" (UniqueName: \"kubernetes.io/projected/2c3b6f02-a256-4458-995d-48e43046e73f-kube-api-access-jgg72\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.347223 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.347188    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.347368 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.347233    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.347368 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.347350    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jgg72\" (UniqueName: \"kubernetes.io/projected/2c3b6f02-a256-4458-995d-48e43046e73f-kube-api-access-jgg72\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.347609 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.347587    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.347693 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.347642    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.356027 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.356005    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgg72\" (UniqueName: \"kubernetes.io/projected/2c3b6f02-a256-4458-995d-48e43046e73f-kube-api-access-jgg72\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.468654 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.468594    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:41.600816 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.600767    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"]
May 28 15:14:41.603527 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:14:41.603493    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3b6f02_a256_4458_995d_48e43046e73f.slice/crio-5d5c925a01dac23da49febe75c6a2356a41693fd1fbf8957c1d0f616b7c3d028 WatchSource:0}: Error finding container 5d5c925a01dac23da49febe75c6a2356a41693fd1fbf8957c1d0f616b7c3d028: Status 404 returned error can't find the container with id 5d5c925a01dac23da49febe75c6a2356a41693fd1fbf8957c1d0f616b7c3d028
May 28 15:14:41.956501 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.956463    2560 generic.go:358] "Generic (PLEG): container finished" podID="2c3b6f02-a256-4458-995d-48e43046e73f" containerID="a510c5dbb9740d59e0097ab5b130d077c8eee8381a2ee1dfd3ae3ab9559cc994" exitCode=0
May 28 15:14:41.956691 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.956570    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9" event={"ID":"2c3b6f02-a256-4458-995d-48e43046e73f","Type":"ContainerDied","Data":"a510c5dbb9740d59e0097ab5b130d077c8eee8381a2ee1dfd3ae3ab9559cc994"}
May 28 15:14:41.956691 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:41.956611    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9" event={"ID":"2c3b6f02-a256-4458-995d-48e43046e73f","Type":"ContainerStarted","Data":"5d5c925a01dac23da49febe75c6a2356a41693fd1fbf8957c1d0f616b7c3d028"}
May 28 15:14:42.743538 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.743460    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"]
May 28 15:14:42.746444 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.746425    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:42.749168 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.749113    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 28 15:14:42.750380 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.750359    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 28 15:14:42.750499 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.750357    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
May 28 15:14:42.750499 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.750416    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zk9v5\""
May 28 15:14:42.750499 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.750414    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 28 15:14:42.756784 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.756759    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"]
May 28 15:14:42.860689 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.860660    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt4k5\" (UniqueName: \"kubernetes.io/projected/5690bd1c-ecce-4cc1-a862-6828e861dff2-kube-api-access-zt4k5\") pod \"kube-auth-proxy-79b6697f59-48n6d\" (UID: \"5690bd1c-ecce-4cc1-a862-6828e861dff2\") " pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:42.860853 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.860718    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5690bd1c-ecce-4cc1-a862-6828e861dff2-tls-certs\") pod \"kube-auth-proxy-79b6697f59-48n6d\" (UID: \"5690bd1c-ecce-4cc1-a862-6828e861dff2\") " pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:42.860853 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.860772    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5690bd1c-ecce-4cc1-a862-6828e861dff2-tmp\") pod \"kube-auth-proxy-79b6697f59-48n6d\" (UID: \"5690bd1c-ecce-4cc1-a862-6828e861dff2\") " pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:42.961213 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.961166    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5690bd1c-ecce-4cc1-a862-6828e861dff2-tls-certs\") pod \"kube-auth-proxy-79b6697f59-48n6d\" (UID: \"5690bd1c-ecce-4cc1-a862-6828e861dff2\") " pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:42.961291 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.961229    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5690bd1c-ecce-4cc1-a862-6828e861dff2-tmp\") pod \"kube-auth-proxy-79b6697f59-48n6d\" (UID: \"5690bd1c-ecce-4cc1-a862-6828e861dff2\") " pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:42.961291 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.961273    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zt4k5\" (UniqueName: \"kubernetes.io/projected/5690bd1c-ecce-4cc1-a862-6828e861dff2-kube-api-access-zt4k5\") pod \"kube-auth-proxy-79b6697f59-48n6d\" (UID: \"5690bd1c-ecce-4cc1-a862-6828e861dff2\") " pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:42.961792 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.961618    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9" event={"ID":"2c3b6f02-a256-4458-995d-48e43046e73f","Type":"ContainerStarted","Data":"4e9c6954ef7d5f18195cee64d8c501d4b5902e179420e8c4e071a5cebe45c2f5"}
May 28 15:14:42.963541 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.963521    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5690bd1c-ecce-4cc1-a862-6828e861dff2-tls-certs\") pod \"kube-auth-proxy-79b6697f59-48n6d\" (UID: \"5690bd1c-ecce-4cc1-a862-6828e861dff2\") " pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:42.963700 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.963684    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5690bd1c-ecce-4cc1-a862-6828e861dff2-tmp\") pod \"kube-auth-proxy-79b6697f59-48n6d\" (UID: \"5690bd1c-ecce-4cc1-a862-6828e861dff2\") " pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:42.969123 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:42.969102    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt4k5\" (UniqueName: \"kubernetes.io/projected/5690bd1c-ecce-4cc1-a862-6828e861dff2-kube-api-access-zt4k5\") pod \"kube-auth-proxy-79b6697f59-48n6d\" (UID: \"5690bd1c-ecce-4cc1-a862-6828e861dff2\") " pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:43.057485 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:43.057463    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"
May 28 15:14:43.181398 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:43.181373    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79b6697f59-48n6d"]
May 28 15:14:43.183720 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:14:43.183689    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5690bd1c_ecce_4cc1_a862_6828e861dff2.slice/crio-ce2d8bacd731ae6b71db80730d7892cd45651542b54e6b5df415a62b75fe2e6f WatchSource:0}: Error finding container ce2d8bacd731ae6b71db80730d7892cd45651542b54e6b5df415a62b75fe2e6f: Status 404 returned error can't find the container with id ce2d8bacd731ae6b71db80730d7892cd45651542b54e6b5df415a62b75fe2e6f
May 28 15:14:43.966250 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:43.966213    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d" event={"ID":"5690bd1c-ecce-4cc1-a862-6828e861dff2","Type":"ContainerStarted","Data":"ce2d8bacd731ae6b71db80730d7892cd45651542b54e6b5df415a62b75fe2e6f"}
May 28 15:14:43.968712 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:43.968681    2560 generic.go:358] "Generic (PLEG): container finished" podID="2c3b6f02-a256-4458-995d-48e43046e73f" containerID="4e9c6954ef7d5f18195cee64d8c501d4b5902e179420e8c4e071a5cebe45c2f5" exitCode=0
May 28 15:14:43.968865 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:43.968722    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9" event={"ID":"2c3b6f02-a256-4458-995d-48e43046e73f","Type":"ContainerDied","Data":"4e9c6954ef7d5f18195cee64d8c501d4b5902e179420e8c4e071a5cebe45c2f5"}
May 28 15:14:44.923895 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:44.923858    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-ll6lg"]
May 28 15:14:44.932696 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:44.932669    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:44.934043 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:44.934014    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-ll6lg"]
May 28 15:14:44.935507 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:44.935483    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7zslp\""
May 28 15:14:44.935674 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:44.935604    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
May 28 15:14:44.974098 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:44.974057    2560 generic.go:358] "Generic (PLEG): container finished" podID="2c3b6f02-a256-4458-995d-48e43046e73f" containerID="9a0c93e97d56fa411fd282881fac177ef767bfd63691a819ff796f1a4462fc09" exitCode=0
May 28 15:14:44.974450 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:44.974168    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9" event={"ID":"2c3b6f02-a256-4458-995d-48e43046e73f","Type":"ContainerDied","Data":"9a0c93e97d56fa411fd282881fac177ef767bfd63691a819ff796f1a4462fc09"}
May 28 15:14:44.976620 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:44.976587    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndvlm\" (UniqueName: \"kubernetes.io/projected/c36eea84-6483-4c02-b81e-9d95cf259cf0-kube-api-access-ndvlm\") pod \"odh-model-controller-858dbf95b8-ll6lg\" (UID: \"c36eea84-6483-4c02-b81e-9d95cf259cf0\") " pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:44.976739 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:44.976691    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c36eea84-6483-4c02-b81e-9d95cf259cf0-cert\") pod \"odh-model-controller-858dbf95b8-ll6lg\" (UID: \"c36eea84-6483-4c02-b81e-9d95cf259cf0\") " pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:45.078044 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:45.078011    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c36eea84-6483-4c02-b81e-9d95cf259cf0-cert\") pod \"odh-model-controller-858dbf95b8-ll6lg\" (UID: \"c36eea84-6483-4c02-b81e-9d95cf259cf0\") " pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:45.078230 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:45.078087    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ndvlm\" (UniqueName: \"kubernetes.io/projected/c36eea84-6483-4c02-b81e-9d95cf259cf0-kube-api-access-ndvlm\") pod \"odh-model-controller-858dbf95b8-ll6lg\" (UID: \"c36eea84-6483-4c02-b81e-9d95cf259cf0\") " pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:45.078230 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:14:45.078199    2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
May 28 15:14:45.078314 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:14:45.078283    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c36eea84-6483-4c02-b81e-9d95cf259cf0-cert podName:c36eea84-6483-4c02-b81e-9d95cf259cf0 nodeName:}" failed. No retries permitted until 2026-05-28 15:14:45.578258017 +0000 UTC m=+422.363408661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c36eea84-6483-4c02-b81e-9d95cf259cf0-cert") pod "odh-model-controller-858dbf95b8-ll6lg" (UID: "c36eea84-6483-4c02-b81e-9d95cf259cf0") : secret "odh-model-controller-webhook-cert" not found
May 28 15:14:45.087503 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:45.087474    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndvlm\" (UniqueName: \"kubernetes.io/projected/c36eea84-6483-4c02-b81e-9d95cf259cf0-kube-api-access-ndvlm\") pod \"odh-model-controller-858dbf95b8-ll6lg\" (UID: \"c36eea84-6483-4c02-b81e-9d95cf259cf0\") " pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:45.581709 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:45.581668    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c36eea84-6483-4c02-b81e-9d95cf259cf0-cert\") pod \"odh-model-controller-858dbf95b8-ll6lg\" (UID: \"c36eea84-6483-4c02-b81e-9d95cf259cf0\") " pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:45.584341 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:45.584308    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c36eea84-6483-4c02-b81e-9d95cf259cf0-cert\") pod \"odh-model-controller-858dbf95b8-ll6lg\" (UID: \"c36eea84-6483-4c02-b81e-9d95cf259cf0\") " pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:45.845283 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:45.845189    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:45.970775 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:45.970752    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-ll6lg"]
May 28 15:14:45.974736 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:14:45.974692    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc36eea84_6483_4c02_b81e_9d95cf259cf0.slice/crio-317309a6bf1ecca4b8f96aa5e28b71ffc1d014a9b2bc816c00d58a6b92288980 WatchSource:0}: Error finding container 317309a6bf1ecca4b8f96aa5e28b71ffc1d014a9b2bc816c00d58a6b92288980: Status 404 returned error can't find the container with id 317309a6bf1ecca4b8f96aa5e28b71ffc1d014a9b2bc816c00d58a6b92288980
May 28 15:14:46.087815 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.087781    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:46.187240 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.187158    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-bundle\") pod \"2c3b6f02-a256-4458-995d-48e43046e73f\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") "
May 28 15:14:46.187240 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.187207    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgg72\" (UniqueName: \"kubernetes.io/projected/2c3b6f02-a256-4458-995d-48e43046e73f-kube-api-access-jgg72\") pod \"2c3b6f02-a256-4458-995d-48e43046e73f\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") "
May 28 15:14:46.187508 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.187258    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-util\") pod \"2c3b6f02-a256-4458-995d-48e43046e73f\" (UID: \"2c3b6f02-a256-4458-995d-48e43046e73f\") "
May 28 15:14:46.188259 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.188212    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-bundle" (OuterVolumeSpecName: "bundle") pod "2c3b6f02-a256-4458-995d-48e43046e73f" (UID: "2c3b6f02-a256-4458-995d-48e43046e73f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:14:46.189794 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.189766    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c3b6f02-a256-4458-995d-48e43046e73f-kube-api-access-jgg72" (OuterVolumeSpecName: "kube-api-access-jgg72") pod "2c3b6f02-a256-4458-995d-48e43046e73f" (UID: "2c3b6f02-a256-4458-995d-48e43046e73f"). InnerVolumeSpecName "kube-api-access-jgg72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:14:46.193221 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.193186    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-util" (OuterVolumeSpecName: "util") pod "2c3b6f02-a256-4458-995d-48e43046e73f" (UID: "2c3b6f02-a256-4458-995d-48e43046e73f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:14:46.288400 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.288360    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-bundle\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:14:46.288400 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.288398    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgg72\" (UniqueName: \"kubernetes.io/projected/2c3b6f02-a256-4458-995d-48e43046e73f-kube-api-access-jgg72\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:14:46.288647 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.288416    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c3b6f02-a256-4458-995d-48e43046e73f-util\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:14:46.808856 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.808836    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 28 15:14:46.982863 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.982821    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg" event={"ID":"c36eea84-6483-4c02-b81e-9d95cf259cf0","Type":"ContainerStarted","Data":"317309a6bf1ecca4b8f96aa5e28b71ffc1d014a9b2bc816c00d58a6b92288980"}
May 28 15:14:46.984823 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.984795    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9"
May 28 15:14:46.984932 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.984798    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c48355hpx9" event={"ID":"2c3b6f02-a256-4458-995d-48e43046e73f","Type":"ContainerDied","Data":"5d5c925a01dac23da49febe75c6a2356a41693fd1fbf8957c1d0f616b7c3d028"}
May 28 15:14:46.984932 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.984909    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d5c925a01dac23da49febe75c6a2356a41693fd1fbf8957c1d0f616b7c3d028"
May 28 15:14:46.986734 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:46.986702    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d" event={"ID":"5690bd1c-ecce-4cc1-a862-6828e861dff2","Type":"ContainerStarted","Data":"69bf542424681fbdebd366423f23ba7355720d3edbbc74148f644b7d345d53d8"}
May 28 15:14:47.002849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:47.002802    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79b6697f59-48n6d" podStartSLOduration=1.382108495 podStartE2EDuration="5.002786566s" podCreationTimestamp="2026-05-28 15:14:42 +0000 UTC" firstStartedPulling="2026-05-28 15:14:43.185435731 +0000 UTC m=+419.970586369" lastFinishedPulling="2026-05-28 15:14:46.806113793 +0000 UTC m=+423.591264440" observedRunningTime="2026-05-28 15:14:47.001934241 +0000 UTC m=+423.787084902" watchObservedRunningTime="2026-05-28 15:14:47.002786566 +0000 UTC m=+423.787937227"
May 28 15:14:50.003325 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.003284    2560 generic.go:358] "Generic (PLEG): container finished" podID="c36eea84-6483-4c02-b81e-9d95cf259cf0" containerID="18ccbe4a6854a8cf261360a863b28b58db22ba74d7233597d07d31e461302143" exitCode=1
May 28 15:14:50.003838 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.003354    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg" event={"ID":"c36eea84-6483-4c02-b81e-9d95cf259cf0","Type":"ContainerDied","Data":"18ccbe4a6854a8cf261360a863b28b58db22ba74d7233597d07d31e461302143"}
May 28 15:14:50.003838 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.003630    2560 scope.go:117] "RemoveContainer" containerID="18ccbe4a6854a8cf261360a863b28b58db22ba74d7233597d07d31e461302143"
May 28 15:14:50.126519 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.126486    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-8vgmq"]
May 28 15:14:50.126768 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.126756    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3b6f02-a256-4458-995d-48e43046e73f" containerName="util"
May 28 15:14:50.126811 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.126770    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3b6f02-a256-4458-995d-48e43046e73f" containerName="util"
May 28 15:14:50.126811 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.126786    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3b6f02-a256-4458-995d-48e43046e73f" containerName="pull"
May 28 15:14:50.126811 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.126791    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3b6f02-a256-4458-995d-48e43046e73f" containerName="pull"
May 28 15:14:50.126811 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.126797    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3b6f02-a256-4458-995d-48e43046e73f" containerName="extract"
May 28 15:14:50.126811 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.126803    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3b6f02-a256-4458-995d-48e43046e73f" containerName="extract"
May 28 15:14:50.126951 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.126843    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="2c3b6f02-a256-4458-995d-48e43046e73f" containerName="extract"
May 28 15:14:50.129632 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.129616    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:50.132419 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.132398    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
May 28 15:14:50.132591 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.132573    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tg6lq\""
May 28 15:14:50.139248 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.139223    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-8vgmq"]
May 28 15:14:50.220026 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.219998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c73378a5-13ea-4643-aa94-0f2f0fe08550-cert\") pod \"kserve-controller-manager-856948b99f-8vgmq\" (UID: \"c73378a5-13ea-4643-aa94-0f2f0fe08550\") " pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:50.220179 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.220033    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlmj5\" (UniqueName: \"kubernetes.io/projected/c73378a5-13ea-4643-aa94-0f2f0fe08550-kube-api-access-qlmj5\") pod \"kserve-controller-manager-856948b99f-8vgmq\" (UID: \"c73378a5-13ea-4643-aa94-0f2f0fe08550\") " pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:50.320721 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.320692    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c73378a5-13ea-4643-aa94-0f2f0fe08550-cert\") pod \"kserve-controller-manager-856948b99f-8vgmq\" (UID: \"c73378a5-13ea-4643-aa94-0f2f0fe08550\") " pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:50.320915 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.320735    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qlmj5\" (UniqueName: \"kubernetes.io/projected/c73378a5-13ea-4643-aa94-0f2f0fe08550-kube-api-access-qlmj5\") pod \"kserve-controller-manager-856948b99f-8vgmq\" (UID: \"c73378a5-13ea-4643-aa94-0f2f0fe08550\") " pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:50.320915 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:14:50.320851    2560 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
May 28 15:14:50.321020 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:14:50.320934    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c73378a5-13ea-4643-aa94-0f2f0fe08550-cert podName:c73378a5-13ea-4643-aa94-0f2f0fe08550 nodeName:}" failed. No retries permitted until 2026-05-28 15:14:50.820918229 +0000 UTC m=+427.606068867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c73378a5-13ea-4643-aa94-0f2f0fe08550-cert") pod "kserve-controller-manager-856948b99f-8vgmq" (UID: "c73378a5-13ea-4643-aa94-0f2f0fe08550") : secret "kserve-webhook-server-cert" not found
May 28 15:14:50.336134 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.336110    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlmj5\" (UniqueName: \"kubernetes.io/projected/c73378a5-13ea-4643-aa94-0f2f0fe08550-kube-api-access-qlmj5\") pod \"kserve-controller-manager-856948b99f-8vgmq\" (UID: \"c73378a5-13ea-4643-aa94-0f2f0fe08550\") " pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:50.824519 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.824485    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c73378a5-13ea-4643-aa94-0f2f0fe08550-cert\") pod \"kserve-controller-manager-856948b99f-8vgmq\" (UID: \"c73378a5-13ea-4643-aa94-0f2f0fe08550\") " pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:50.827182 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:50.827154    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c73378a5-13ea-4643-aa94-0f2f0fe08550-cert\") pod \"kserve-controller-manager-856948b99f-8vgmq\" (UID: \"c73378a5-13ea-4643-aa94-0f2f0fe08550\") " pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:51.008305 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:51.008274    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg" event={"ID":"c36eea84-6483-4c02-b81e-9d95cf259cf0","Type":"ContainerStarted","Data":"f9874b694d31a98836575a61d68b6b28329ec0179227fb781199871dc244bb66"}
May 28 15:14:51.008768 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:51.008415    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:14:51.026659 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:51.026615    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg" podStartSLOduration=2.597748825 podStartE2EDuration="7.026602948s" podCreationTimestamp="2026-05-28 15:14:44 +0000 UTC" firstStartedPulling="2026-05-28 15:14:45.976002226 +0000 UTC m=+422.761152864" lastFinishedPulling="2026-05-28 15:14:50.404856344 +0000 UTC m=+427.190006987" observedRunningTime="2026-05-28 15:14:51.026298339 +0000 UTC m=+427.811449000" watchObservedRunningTime="2026-05-28 15:14:51.026602948 +0000 UTC m=+427.811753648"
May 28 15:14:51.040413 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:51.040390    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:51.167049 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:51.167009    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-8vgmq"]
May 28 15:14:51.170137 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:14:51.170109    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73378a5_13ea_4643_aa94_0f2f0fe08550.slice/crio-ff66206e94534ef743806dbc57190522ed254dc7860a998b5f4f76dbe83f8565 WatchSource:0}: Error finding container ff66206e94534ef743806dbc57190522ed254dc7860a998b5f4f76dbe83f8565: Status 404 returned error can't find the container with id ff66206e94534ef743806dbc57190522ed254dc7860a998b5f4f76dbe83f8565
May 28 15:14:52.013182 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:52.013137    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq" event={"ID":"c73378a5-13ea-4643-aa94-0f2f0fe08550","Type":"ContainerStarted","Data":"ff66206e94534ef743806dbc57190522ed254dc7860a998b5f4f76dbe83f8565"}
May 28 15:14:55.024914 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.024876    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq" event={"ID":"c73378a5-13ea-4643-aa94-0f2f0fe08550","Type":"ContainerStarted","Data":"b0d1b13b84dc3a139bb10d46dc2a7e18c01eabd2cc3a66f594ee65de10bbaae9"}
May 28 15:14:55.025299 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.025061    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:14:55.047102 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.047056    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq" podStartSLOduration=1.300636033 podStartE2EDuration="5.047044383s" podCreationTimestamp="2026-05-28 15:14:50 +0000 UTC" firstStartedPulling="2026-05-28 15:14:51.172054297 +0000 UTC m=+427.957204953" lastFinishedPulling="2026-05-28 15:14:54.918462665 +0000 UTC m=+431.703613303" observedRunningTime="2026-05-28 15:14:55.045463342 +0000 UTC m=+431.830614002" watchObservedRunningTime="2026-05-28 15:14:55.047044383 +0000 UTC m=+431.832195043"
May 28 15:14:55.559046 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.559013    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"]
May 28 15:14:55.562253 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.562236    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.565161 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.565133    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ldgjz\""
May 28 15:14:55.565287 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.565181    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 28 15:14:55.566215 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.566200    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 28 15:14:55.576350 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.576327    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"]
May 28 15:14:55.661602 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.661576    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.661737 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.661614    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.661737 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.661656    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fn6k\" (UniqueName: \"kubernetes.io/projected/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-kube-api-access-2fn6k\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.762687 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.762656    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.762687 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.762690    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.762872 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.762740    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2fn6k\" (UniqueName: \"kubernetes.io/projected/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-kube-api-access-2fn6k\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.763033 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.763014    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.763071 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.763039    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.772216 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.772186    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fn6k\" (UniqueName: \"kubernetes.io/projected/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-kube-api-access-2fn6k\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:55.872055 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:55.871977    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:14:56.029010 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.028976    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"]
May 28 15:14:56.035393 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:14:56.035356    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5628dc_f5fa_4544_bdf3_4ae8584dafbc.slice/crio-d2a34376d54b36bd141ed56ab4fea20eded2dfad7d5d7b082918bc0b682a6dfb WatchSource:0}: Error finding container d2a34376d54b36bd141ed56ab4fea20eded2dfad7d5d7b082918bc0b682a6dfb: Status 404 returned error can't find the container with id d2a34376d54b36bd141ed56ab4fea20eded2dfad7d5d7b082918bc0b682a6dfb
May 28 15:14:56.792867 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.792832    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"]
May 28 15:14:56.796304 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.796286    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:14:56.798990 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.798965    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
May 28 15:14:56.799111 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.799007    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
May 28 15:14:56.799111 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.798974    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-lgv42\""
May 28 15:14:56.808997 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.808975    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"]
May 28 15:14:56.871432 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.871405    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/d81c9ce3-7ef3-4ef0-bab3-821167837476-operator-config\") pod \"servicemesh-operator3-55f49c5f94-4gmbr\" (UID: \"d81c9ce3-7ef3-4ef0-bab3-821167837476\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:14:56.871589 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.871444    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh4k8\" (UniqueName: \"kubernetes.io/projected/d81c9ce3-7ef3-4ef0-bab3-821167837476-kube-api-access-lh4k8\") pod \"servicemesh-operator3-55f49c5f94-4gmbr\" (UID: \"d81c9ce3-7ef3-4ef0-bab3-821167837476\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:14:56.972273 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.972236    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/d81c9ce3-7ef3-4ef0-bab3-821167837476-operator-config\") pod \"servicemesh-operator3-55f49c5f94-4gmbr\" (UID: \"d81c9ce3-7ef3-4ef0-bab3-821167837476\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:14:56.972404 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.972284    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh4k8\" (UniqueName: \"kubernetes.io/projected/d81c9ce3-7ef3-4ef0-bab3-821167837476-kube-api-access-lh4k8\") pod \"servicemesh-operator3-55f49c5f94-4gmbr\" (UID: \"d81c9ce3-7ef3-4ef0-bab3-821167837476\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:14:56.975050 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.975022    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/d81c9ce3-7ef3-4ef0-bab3-821167837476-operator-config\") pod \"servicemesh-operator3-55f49c5f94-4gmbr\" (UID: \"d81c9ce3-7ef3-4ef0-bab3-821167837476\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:14:56.980860 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:56.980836    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh4k8\" (UniqueName: \"kubernetes.io/projected/d81c9ce3-7ef3-4ef0-bab3-821167837476-kube-api-access-lh4k8\") pod \"servicemesh-operator3-55f49c5f94-4gmbr\" (UID: \"d81c9ce3-7ef3-4ef0-bab3-821167837476\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:14:57.033631 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:57.033590    2560 generic.go:358] "Generic (PLEG): container finished" podID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerID="f32dac23cd6189205baefcf92adea93ba163da3387360d701e00fa0ea2e65d1e" exitCode=0
May 28 15:14:57.034063 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:57.033716    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q" event={"ID":"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc","Type":"ContainerDied","Data":"f32dac23cd6189205baefcf92adea93ba163da3387360d701e00fa0ea2e65d1e"}
May 28 15:14:57.034063 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:57.033744    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q" event={"ID":"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc","Type":"ContainerStarted","Data":"d2a34376d54b36bd141ed56ab4fea20eded2dfad7d5d7b082918bc0b682a6dfb"}
May 28 15:14:57.105532 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:57.105469    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:14:57.231927 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:57.231882    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"]
May 28 15:14:57.234146 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:14:57.234107    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81c9ce3_7ef3_4ef0_bab3_821167837476.slice/crio-89f71a2376f57e1b55765f558ce573d847bd8c565d35b7f46c530872ce9a4fe2 WatchSource:0}: Error finding container 89f71a2376f57e1b55765f558ce573d847bd8c565d35b7f46c530872ce9a4fe2: Status 404 returned error can't find the container with id 89f71a2376f57e1b55765f558ce573d847bd8c565d35b7f46c530872ce9a4fe2
May 28 15:14:58.038124 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:58.038086    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr" event={"ID":"d81c9ce3-7ef3-4ef0-bab3-821167837476","Type":"ContainerStarted","Data":"89f71a2376f57e1b55765f558ce573d847bd8c565d35b7f46c530872ce9a4fe2"}
May 28 15:14:59.043666 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:59.043626    2560 generic.go:358] "Generic (PLEG): container finished" podID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerID="a8d6752962576ddd444f6f56627935a1bea32da7cef449a6b4fcf400f78b6e99" exitCode=0
May 28 15:14:59.044124 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:14:59.043671    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q" event={"ID":"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc","Type":"ContainerDied","Data":"a8d6752962576ddd444f6f56627935a1bea32da7cef449a6b4fcf400f78b6e99"}
May 28 15:15:00.049043 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:00.049011    2560 generic.go:358] "Generic (PLEG): container finished" podID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerID="6dddfeb68467d934dc4629c259a4c600c6e8ebdfbe19eec81048fd126379c780" exitCode=0
May 28 15:15:00.049399 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:00.049074    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q" event={"ID":"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc","Type":"ContainerDied","Data":"6dddfeb68467d934dc4629c259a4c600c6e8ebdfbe19eec81048fd126379c780"}
May 28 15:15:01.823456 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:01.823434    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:15:01.910753 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:01.910731    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fn6k\" (UniqueName: \"kubernetes.io/projected/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-kube-api-access-2fn6k\") pod \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") "
May 28 15:15:01.910849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:01.910794    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-bundle\") pod \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") "
May 28 15:15:01.910849 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:01.910843    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-util\") pod \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\" (UID: \"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc\") "
May 28 15:15:01.912025 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:01.911998    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-bundle" (OuterVolumeSpecName: "bundle") pod "6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" (UID: "6c5628dc-f5fa-4544-bdf3-4ae8584dafbc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:15:01.913002 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:01.912974    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-kube-api-access-2fn6k" (OuterVolumeSpecName: "kube-api-access-2fn6k") pod "6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" (UID: "6c5628dc-f5fa-4544-bdf3-4ae8584dafbc"). InnerVolumeSpecName "kube-api-access-2fn6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:15:01.916822 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:01.916781    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-util" (OuterVolumeSpecName: "util") pod "6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" (UID: "6c5628dc-f5fa-4544-bdf3-4ae8584dafbc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:15:02.012248 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.012171    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-util\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:15:02.012248 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.012210    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fn6k\" (UniqueName: \"kubernetes.io/projected/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-kube-api-access-2fn6k\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:15:02.012248 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.012224    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c5628dc-f5fa-4544-bdf3-4ae8584dafbc-bundle\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:15:02.015726 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.015673    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-ll6lg"
May 28 15:15:02.059154 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.059073    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q"
May 28 15:15:02.059680 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.059321    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avtx6q" event={"ID":"6c5628dc-f5fa-4544-bdf3-4ae8584dafbc","Type":"ContainerDied","Data":"d2a34376d54b36bd141ed56ab4fea20eded2dfad7d5d7b082918bc0b682a6dfb"}
May 28 15:15:02.059778 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.059695    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2a34376d54b36bd141ed56ab4fea20eded2dfad7d5d7b082918bc0b682a6dfb"
May 28 15:15:02.062845 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.062262    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr" event={"ID":"d81c9ce3-7ef3-4ef0-bab3-821167837476","Type":"ContainerStarted","Data":"dc6af8f2ff1af8c9917b325eb66d112ce40d034c729876908f674632fab37e8e"}
May 28 15:15:02.069708 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.069687    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:15:02.097871 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:02.097820    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr" podStartSLOduration=1.476995719 podStartE2EDuration="6.09779794s" podCreationTimestamp="2026-05-28 15:14:56 +0000 UTC" firstStartedPulling="2026-05-28 15:14:57.236641433 +0000 UTC m=+434.021792071" lastFinishedPulling="2026-05-28 15:15:01.85744365 +0000 UTC m=+438.642594292" observedRunningTime="2026-05-28 15:15:02.096576157 +0000 UTC m=+438.881726809" watchObservedRunningTime="2026-05-28 15:15:02.09779794 +0000 UTC m=+438.882948605"
May 28 15:15:12.299728 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.299690    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"]
May 28 15:15:12.300188 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.300059    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerName="pull"
May 28 15:15:12.300188 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.300074    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerName="pull"
May 28 15:15:12.300188 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.300095    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerName="extract"
May 28 15:15:12.300188 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.300103    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerName="extract"
May 28 15:15:12.300188 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.300118    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerName="util"
May 28 15:15:12.300188 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.300128    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerName="util"
May 28 15:15:12.300369 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.300201    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="6c5628dc-f5fa-4544-bdf3-4ae8584dafbc" containerName="extract"
May 28 15:15:12.302349 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.302328    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.305296 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.305267    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
May 28 15:15:12.305296 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.305265    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
May 28 15:15:12.305480 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.305292    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 28 15:15:12.305480 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.305403    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6nmrq\""
May 28 15:15:12.305480 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.305477    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
May 28 15:15:12.313538 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.313516    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"]
May 28 15:15:12.395283 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.395249    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e211b9a4-02f3-4323-beb7-d2e06737ba30-local-certs\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.395283 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.395282    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rltdx\" (UniqueName: \"kubernetes.io/projected/e211b9a4-02f3-4323-beb7-d2e06737ba30-kube-api-access-rltdx\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.395501 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.395318    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.395501 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.395392    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-token\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.395501 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.395423    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.395501 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.395487    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.395727 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.395533    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e211b9a4-02f3-4323-beb7-d2e06737ba30-cacerts\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.496621 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.496583    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.496621 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.496629    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e211b9a4-02f3-4323-beb7-d2e06737ba30-cacerts\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.496881 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.496653    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e211b9a4-02f3-4323-beb7-d2e06737ba30-local-certs\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.496881 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.496681    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rltdx\" (UniqueName: \"kubernetes.io/projected/e211b9a4-02f3-4323-beb7-d2e06737ba30-kube-api-access-rltdx\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.496881 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.496731    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.496881 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.496770    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-token\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.496881 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.496804    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.497411 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.497369    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.499532 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.499505    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e211b9a4-02f3-4323-beb7-d2e06737ba30-local-certs\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.499676 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.499590    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.499830 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.499805    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e211b9a4-02f3-4323-beb7-d2e06737ba30-cacerts\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.499887 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.499811    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.505119 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.505089    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rltdx\" (UniqueName: \"kubernetes.io/projected/e211b9a4-02f3-4323-beb7-d2e06737ba30-kube-api-access-rltdx\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.505349 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.505330    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e211b9a4-02f3-4323-beb7-d2e06737ba30-istio-token\") pod \"istiod-openshift-gateway-798958bb55-b9lw5\" (UID: \"e211b9a4-02f3-4323-beb7-d2e06737ba30\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.613071 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.612982    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:12.754566 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:12.754525    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"]
May 28 15:15:12.758025 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:15:12.757997    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode211b9a4_02f3_4323_beb7_d2e06737ba30.slice/crio-1333e45f722caf01701b16c371ee4125f3517277e98115757903703e4953e2ab WatchSource:0}: Error finding container 1333e45f722caf01701b16c371ee4125f3517277e98115757903703e4953e2ab: Status 404 returned error can't find the container with id 1333e45f722caf01701b16c371ee4125f3517277e98115757903703e4953e2ab
May 28 15:15:13.101490 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:13.101450    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5" event={"ID":"e211b9a4-02f3-4323-beb7-d2e06737ba30","Type":"ContainerStarted","Data":"1333e45f722caf01701b16c371ee4125f3517277e98115757903703e4953e2ab"}
May 28 15:15:14.072435 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:14.072395    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4gmbr"
May 28 15:15:15.465489 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:15.461975    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 15:15:15.465489 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:15.462062    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 15:15:16.117335 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:16.117300    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5" event={"ID":"e211b9a4-02f3-4323-beb7-d2e06737ba30","Type":"ContainerStarted","Data":"b9bcae0c244065032f5fd4c9eb7f1616fc462085eab85268d02a313f473d4aff"}
May 28 15:15:16.117538 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:16.117459    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:16.119171 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:16.119140    2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-b9lw5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
May 28 15:15:16.119294 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:16.119198    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5" podUID="e211b9a4-02f3-4323-beb7-d2e06737ba30" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 28 15:15:16.138131 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:16.138063    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5" podStartSLOduration=1.436989609 podStartE2EDuration="4.138049759s" podCreationTimestamp="2026-05-28 15:15:12 +0000 UTC" firstStartedPulling="2026-05-28 15:15:12.760641277 +0000 UTC m=+449.545791934" lastFinishedPulling="2026-05-28 15:15:15.461701445 +0000 UTC m=+452.246852084" observedRunningTime="2026-05-28 15:15:16.136184181 +0000 UTC m=+452.921334843" watchObservedRunningTime="2026-05-28 15:15:16.138049759 +0000 UTC m=+452.923200419"
May 28 15:15:17.121692 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:17.121662    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-b9lw5"
May 28 15:15:26.033786 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:26.033757    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-8vgmq"
May 28 15:15:56.470063 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.470028    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"]
May 28 15:15:56.473371 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.473353    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.476295 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.476270    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 28 15:15:56.476417 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.476302    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 28 15:15:56.477464 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.477448    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jxjrw\""
May 28 15:15:56.481907 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.481884    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"]
May 28 15:15:56.535832 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.535799    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kcs4\" (UniqueName: \"kubernetes.io/projected/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-kube-api-access-2kcs4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.535948 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.535839    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.535948 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.535856    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.636869 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.636839    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcs4\" (UniqueName: \"kubernetes.io/projected/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-kube-api-access-2kcs4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.637010 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.636890    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.637010 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.636922    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.637314 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.637292    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.637402 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.637320    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.646749 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.646727    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kcs4\" (UniqueName: \"kubernetes.io/projected/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-kube-api-access-2kcs4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.783767 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.783691    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:15:56.918794 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:56.918762    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"]
May 28 15:15:56.920155 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:15:56.920114    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f38cb4_fab9_4e4b_b700_8562a6e74b35.slice/crio-0cdb199d6bfd454c1a195e1c743bd9665bd1759885ed9b9892b509cb6b9cc6cc WatchSource:0}: Error finding container 0cdb199d6bfd454c1a195e1c743bd9665bd1759885ed9b9892b509cb6b9cc6cc: Status 404 returned error can't find the container with id 0cdb199d6bfd454c1a195e1c743bd9665bd1759885ed9b9892b509cb6b9cc6cc
May 28 15:15:57.256271 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:57.256230    2560 generic.go:358] "Generic (PLEG): container finished" podID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerID="d4723001ede810c24b3bf822cb2687a556551fccf7b565af49e974276a2420d8" exitCode=0
May 28 15:15:57.256427 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:57.256322    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb" event={"ID":"b6f38cb4-fab9-4e4b-b700-8562a6e74b35","Type":"ContainerDied","Data":"d4723001ede810c24b3bf822cb2687a556551fccf7b565af49e974276a2420d8"}
May 28 15:15:57.256427 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:57.256354    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb" event={"ID":"b6f38cb4-fab9-4e4b-b700-8562a6e74b35","Type":"ContainerStarted","Data":"0cdb199d6bfd454c1a195e1c743bd9665bd1759885ed9b9892b509cb6b9cc6cc"}
May 28 15:15:58.261871 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:58.261841    2560 generic.go:358] "Generic (PLEG): container finished" podID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerID="2abd635ac383934ff47791b9af3cd12b9e45e7493aa51d9bcdc9bbd2c834dbdc" exitCode=0
May 28 15:15:58.262326 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:58.261924    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb" event={"ID":"b6f38cb4-fab9-4e4b-b700-8562a6e74b35","Type":"ContainerDied","Data":"2abd635ac383934ff47791b9af3cd12b9e45e7493aa51d9bcdc9bbd2c834dbdc"}
May 28 15:15:59.266924 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:59.266882    2560 generic.go:358] "Generic (PLEG): container finished" podID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerID="922796be248540ef64f6ecbbc5c990da7a768d62ce048f2fcd0a1370571d19cd" exitCode=0
May 28 15:15:59.267327 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:15:59.266960    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb" event={"ID":"b6f38cb4-fab9-4e4b-b700-8562a6e74b35","Type":"ContainerDied","Data":"922796be248540ef64f6ecbbc5c990da7a768d62ce048f2fcd0a1370571d19cd"}
May 28 15:16:00.393035 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.393012    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:16:00.463359 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.463313    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kcs4\" (UniqueName: \"kubernetes.io/projected/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-kube-api-access-2kcs4\") pod \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") "
May 28 15:16:00.463359 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.463366    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-bundle\") pod \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") "
May 28 15:16:00.463641 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.463418    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-util\") pod \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\" (UID: \"b6f38cb4-fab9-4e4b-b700-8562a6e74b35\") "
May 28 15:16:00.463968 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.463939    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-bundle" (OuterVolumeSpecName: "bundle") pod "b6f38cb4-fab9-4e4b-b700-8562a6e74b35" (UID: "b6f38cb4-fab9-4e4b-b700-8562a6e74b35"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:16:00.465650 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.465617    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-kube-api-access-2kcs4" (OuterVolumeSpecName: "kube-api-access-2kcs4") pod "b6f38cb4-fab9-4e4b-b700-8562a6e74b35" (UID: "b6f38cb4-fab9-4e4b-b700-8562a6e74b35"). InnerVolumeSpecName "kube-api-access-2kcs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:16:00.469058 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.469027    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-util" (OuterVolumeSpecName: "util") pod "b6f38cb4-fab9-4e4b-b700-8562a6e74b35" (UID: "b6f38cb4-fab9-4e4b-b700-8562a6e74b35"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 15:16:00.564497 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.564472    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-util\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:16:00.564497 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.564496    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kcs4\" (UniqueName: \"kubernetes.io/projected/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-kube-api-access-2kcs4\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:16:00.564712 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:00.564507    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6f38cb4-fab9-4e4b-b700-8562a6e74b35-bundle\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:16:01.275241 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:01.275209    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb" event={"ID":"b6f38cb4-fab9-4e4b-b700-8562a6e74b35","Type":"ContainerDied","Data":"0cdb199d6bfd454c1a195e1c743bd9665bd1759885ed9b9892b509cb6b9cc6cc"}
May 28 15:16:01.275241 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:01.275242    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb"
May 28 15:16:01.275433 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:01.275246    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cdb199d6bfd454c1a195e1c743bd9665bd1759885ed9b9892b509cb6b9cc6cc"
May 28 15:16:13.769842 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.769807    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"]
May 28 15:16:13.770425 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.770230    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerName="util"
May 28 15:16:13.770425 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.770247    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerName="util"
May 28 15:16:13.770425 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.770261    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerName="extract"
May 28 15:16:13.770425 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.770271    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerName="extract"
May 28 15:16:13.770425 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.770284    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerName="pull"
May 28 15:16:13.770425 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.770293    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerName="pull"
May 28 15:16:13.770425 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.770375    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="b6f38cb4-fab9-4e4b-b700-8562a6e74b35" containerName="extract"
May 28 15:16:13.774442 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.774419    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"
May 28 15:16:13.777962 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.777939    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 28 15:16:13.778850 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.778834    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 28 15:16:13.778937 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.778872    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-t2g78\""
May 28 15:16:13.792752 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.792726    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"]
May 28 15:16:13.871610 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.871577    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82gft\" (UniqueName: \"kubernetes.io/projected/37510b7f-7d29-441f-986e-8c88aa363fb8-kube-api-access-82gft\") pod \"limitador-operator-controller-manager-85c4996f8c-jlhcb\" (UID: \"37510b7f-7d29-441f-986e-8c88aa363fb8\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"
May 28 15:16:13.972430 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.972401    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-82gft\" (UniqueName: \"kubernetes.io/projected/37510b7f-7d29-441f-986e-8c88aa363fb8-kube-api-access-82gft\") pod \"limitador-operator-controller-manager-85c4996f8c-jlhcb\" (UID: \"37510b7f-7d29-441f-986e-8c88aa363fb8\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"
May 28 15:16:13.984380 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:13.984350    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82gft\" (UniqueName: \"kubernetes.io/projected/37510b7f-7d29-441f-986e-8c88aa363fb8-kube-api-access-82gft\") pod \"limitador-operator-controller-manager-85c4996f8c-jlhcb\" (UID: \"37510b7f-7d29-441f-986e-8c88aa363fb8\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"
May 28 15:16:14.090273 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:14.090233    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"
May 28 15:16:14.224448 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:14.224422    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"]
May 28 15:16:14.225993 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:16:14.225953    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37510b7f_7d29_441f_986e_8c88aa363fb8.slice/crio-18badf3e5aeff50e76d4d24c87750cbd97abb05137a3ac9e43155983905aef9b WatchSource:0}: Error finding container 18badf3e5aeff50e76d4d24c87750cbd97abb05137a3ac9e43155983905aef9b: Status 404 returned error can't find the container with id 18badf3e5aeff50e76d4d24c87750cbd97abb05137a3ac9e43155983905aef9b
May 28 15:16:14.319448 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:14.319406    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb" event={"ID":"37510b7f-7d29-441f-986e-8c88aa363fb8","Type":"ContainerStarted","Data":"18badf3e5aeff50e76d4d24c87750cbd97abb05137a3ac9e43155983905aef9b"}
May 28 15:16:17.332959 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:17.332918    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb" event={"ID":"37510b7f-7d29-441f-986e-8c88aa363fb8","Type":"ContainerStarted","Data":"b5a93473097092318bb5cfac33d3abb1e784a1e42b933c7642749fae139f3716"}
May 28 15:16:17.333391 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:17.333025    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"
May 28 15:16:17.350119 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:17.350063    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb" podStartSLOduration=2.090413799 podStartE2EDuration="4.35005074s" podCreationTimestamp="2026-05-28 15:16:13 +0000 UTC" firstStartedPulling="2026-05-28 15:16:14.227882428 +0000 UTC m=+511.013033070" lastFinishedPulling="2026-05-28 15:16:16.487519372 +0000 UTC m=+513.272670011" observedRunningTime="2026-05-28 15:16:17.348962668 +0000 UTC m=+514.134113329" watchObservedRunningTime="2026-05-28 15:16:17.35005074 +0000 UTC m=+514.135201400"
May 28 15:16:21.901887 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:21.901846    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-bzpbv"]
May 28 15:16:21.905123 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:21.905100    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-bzpbv"
May 28 15:16:21.907846 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:21.907825    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wf8vq\""
May 28 15:16:21.919599 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:21.919574    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-bzpbv"]
May 28 15:16:22.037175 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:22.037143    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmbqc\" (UniqueName: \"kubernetes.io/projected/97ecd4c9-d5e2-4003-9a28-de1f09fe7b01-kube-api-access-lmbqc\") pod \"authorino-operator-657f44b778-bzpbv\" (UID: \"97ecd4c9-d5e2-4003-9a28-de1f09fe7b01\") " pod="kuadrant-system/authorino-operator-657f44b778-bzpbv"
May 28 15:16:22.137678 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:22.137646    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lmbqc\" (UniqueName: \"kubernetes.io/projected/97ecd4c9-d5e2-4003-9a28-de1f09fe7b01-kube-api-access-lmbqc\") pod \"authorino-operator-657f44b778-bzpbv\" (UID: \"97ecd4c9-d5e2-4003-9a28-de1f09fe7b01\") " pod="kuadrant-system/authorino-operator-657f44b778-bzpbv"
May 28 15:16:22.147151 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:22.147124    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmbqc\" (UniqueName: \"kubernetes.io/projected/97ecd4c9-d5e2-4003-9a28-de1f09fe7b01-kube-api-access-lmbqc\") pod \"authorino-operator-657f44b778-bzpbv\" (UID: \"97ecd4c9-d5e2-4003-9a28-de1f09fe7b01\") " pod="kuadrant-system/authorino-operator-657f44b778-bzpbv"
May 28 15:16:22.215257 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:22.215189    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-bzpbv"
May 28 15:16:22.344031 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:22.344005    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-bzpbv"]
May 28 15:16:22.345924 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:16:22.345889    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ecd4c9_d5e2_4003_9a28_de1f09fe7b01.slice/crio-05e8954357ac050d8fb76c0d54300fec539dcb411bb8e886366a77d73d9da27d WatchSource:0}: Error finding container 05e8954357ac050d8fb76c0d54300fec539dcb411bb8e886366a77d73d9da27d: Status 404 returned error can't find the container with id 05e8954357ac050d8fb76c0d54300fec539dcb411bb8e886366a77d73d9da27d
May 28 15:16:22.350460 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:22.350435    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-bzpbv" event={"ID":"97ecd4c9-d5e2-4003-9a28-de1f09fe7b01","Type":"ContainerStarted","Data":"05e8954357ac050d8fb76c0d54300fec539dcb411bb8e886366a77d73d9da27d"}
May 28 15:16:25.363008 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:25.362967    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-bzpbv" event={"ID":"97ecd4c9-d5e2-4003-9a28-de1f09fe7b01","Type":"ContainerStarted","Data":"65df5357f0a542ae5a459a8ce3b0648c41db2c87fa6cefd00663aead9b98d95c"}
May 28 15:16:25.363390 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:25.363029    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-bzpbv"
May 28 15:16:25.379532 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:25.379470    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-bzpbv" podStartSLOduration=1.746480969 podStartE2EDuration="4.379453014s" podCreationTimestamp="2026-05-28 15:16:21 +0000 UTC" firstStartedPulling="2026-05-28 15:16:22.348013286 +0000 UTC m=+519.133163925" lastFinishedPulling="2026-05-28 15:16:24.980985332 +0000 UTC m=+521.766135970" observedRunningTime="2026-05-28 15:16:25.378819685 +0000 UTC m=+522.163970346" watchObservedRunningTime="2026-05-28 15:16:25.379453014 +0000 UTC m=+522.164603675"
May 28 15:16:28.339429 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:28.339383    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"
May 28 15:16:36.368739 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:36.368711    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-bzpbv"
May 28 15:16:37.174408 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.174370    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"]
May 28 15:16:37.174756 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.174708    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb" podUID="37510b7f-7d29-441f-986e-8c88aa363fb8" containerName="manager" containerID="cri-o://b5a93473097092318bb5cfac33d3abb1e784a1e42b933c7642749fae139f3716" gracePeriod=2
May 28 15:16:37.180119 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.180089    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"]
May 28 15:16:37.214324 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.214294    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"]
May 28 15:16:37.214818 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.214802    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37510b7f-7d29-441f-986e-8c88aa363fb8" containerName="manager"
May 28 15:16:37.214875 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.214823    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="37510b7f-7d29-441f-986e-8c88aa363fb8" containerName="manager"
May 28 15:16:37.214918 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.214906    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="37510b7f-7d29-441f-986e-8c88aa363fb8" containerName="manager"
May 28 15:16:37.218049 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.218032    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"
May 28 15:16:37.220625 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.220586    2560 status_manager.go:895] "Failed to get status for pod" podUID="37510b7f-7d29-441f-986e-8c88aa363fb8" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb" err="pods \"limitador-operator-controller-manager-85c4996f8c-jlhcb\" is forbidden: User \"system:node:ip-10-0-135-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-61.ec2.internal' and this object"
May 28 15:16:37.229466 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.229448    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"]
May 28 15:16:37.268159 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.268130    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s5zt\" (UniqueName: \"kubernetes.io/projected/cd0c9b04-1571-4705-b2f7-a339b2165519-kube-api-access-9s5zt\") pod \"limitador-operator-controller-manager-85c4996f8c-pbmsx\" (UID: \"cd0c9b04-1571-4705-b2f7-a339b2165519\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"
May 28 15:16:37.369041 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.369004    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9s5zt\" (UniqueName: \"kubernetes.io/projected/cd0c9b04-1571-4705-b2f7-a339b2165519-kube-api-access-9s5zt\") pod \"limitador-operator-controller-manager-85c4996f8c-pbmsx\" (UID: \"cd0c9b04-1571-4705-b2f7-a339b2165519\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"
May 28 15:16:37.383256 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.383222    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s5zt\" (UniqueName: \"kubernetes.io/projected/cd0c9b04-1571-4705-b2f7-a339b2165519-kube-api-access-9s5zt\") pod \"limitador-operator-controller-manager-85c4996f8c-pbmsx\" (UID: \"cd0c9b04-1571-4705-b2f7-a339b2165519\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"
May 28 15:16:37.404421 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.404397    2560 generic.go:358] "Generic (PLEG): container finished" podID="37510b7f-7d29-441f-986e-8c88aa363fb8" containerID="b5a93473097092318bb5cfac33d3abb1e784a1e42b933c7642749fae139f3716" exitCode=0
May 28 15:16:37.404421 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.404416    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"
May 28 15:16:37.404585 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.404464    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18badf3e5aeff50e76d4d24c87750cbd97abb05137a3ac9e43155983905aef9b"
May 28 15:16:37.406847 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.406824    2560 status_manager.go:895] "Failed to get status for pod" podUID="37510b7f-7d29-441f-986e-8c88aa363fb8" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb" err="pods \"limitador-operator-controller-manager-85c4996f8c-jlhcb\" is forbidden: User \"system:node:ip-10-0-135-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-61.ec2.internal' and this object"
May 28 15:16:37.470253 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.470190    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82gft\" (UniqueName: \"kubernetes.io/projected/37510b7f-7d29-441f-986e-8c88aa363fb8-kube-api-access-82gft\") pod \"37510b7f-7d29-441f-986e-8c88aa363fb8\" (UID: \"37510b7f-7d29-441f-986e-8c88aa363fb8\") "
May 28 15:16:37.472331 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.472310    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37510b7f-7d29-441f-986e-8c88aa363fb8-kube-api-access-82gft" (OuterVolumeSpecName: "kube-api-access-82gft") pod "37510b7f-7d29-441f-986e-8c88aa363fb8" (UID: "37510b7f-7d29-441f-986e-8c88aa363fb8"). InnerVolumeSpecName "kube-api-access-82gft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:16:37.550279 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.550252    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"
May 28 15:16:37.571267 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.571245    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82gft\" (UniqueName: \"kubernetes.io/projected/37510b7f-7d29-441f-986e-8c88aa363fb8-kube-api-access-82gft\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:16:37.682863 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.682827    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"]
May 28 15:16:37.685150 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:16:37.685126    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd0c9b04_1571_4705_b2f7_a339b2165519.slice/crio-ba3854bb97f0edb28384b340adc0a8e6f413c1aaf3257709c49b1b1813c7c032 WatchSource:0}: Error finding container ba3854bb97f0edb28384b340adc0a8e6f413c1aaf3257709c49b1b1813c7c032: Status 404 returned error can't find the container with id ba3854bb97f0edb28384b340adc0a8e6f413c1aaf3257709c49b1b1813c7c032
May 28 15:16:37.785267 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:37.785238    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37510b7f-7d29-441f-986e-8c88aa363fb8" path="/var/lib/kubelet/pods/37510b7f-7d29-441f-986e-8c88aa363fb8/volumes"
May 28 15:16:38.409393 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:38.409352    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx" event={"ID":"cd0c9b04-1571-4705-b2f7-a339b2165519","Type":"ContainerStarted","Data":"5eeb41eda610630cc0151cbc5d1977b688852cc3c7c82366a431ba0e7853ab5b"}
May 28 15:16:38.409393 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:38.409372    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-jlhcb"
May 28 15:16:38.409393 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:38.409391    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx" event={"ID":"cd0c9b04-1571-4705-b2f7-a339b2165519","Type":"ContainerStarted","Data":"ba3854bb97f0edb28384b340adc0a8e6f413c1aaf3257709c49b1b1813c7c032"}
May 28 15:16:38.639653 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:38.639600    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx" podStartSLOduration=1.639586862 podStartE2EDuration="1.639586862s" podCreationTimestamp="2026-05-28 15:16:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:16:38.637841691 +0000 UTC m=+535.422992351" watchObservedRunningTime="2026-05-28 15:16:38.639586862 +0000 UTC m=+535.424737521"
May 28 15:16:39.413393 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:39.413365    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"
May 28 15:16:50.418875 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:16:50.418840    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-pbmsx"
May 28 15:18:11.164360 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.164325    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-647584dd76-mnzhh"]
May 28 15:18:11.167460 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.167440    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-647584dd76-mnzhh"
May 28 15:18:11.170051 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.170006    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 28 15:18:11.170051 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.170041    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-b4vq8\""
May 28 15:18:11.175784 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.175760    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-647584dd76-mnzhh"]
May 28 15:18:11.187813 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.187772    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpqfk\" (UniqueName: \"kubernetes.io/projected/a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24-kube-api-access-fpqfk\") pod \"maas-controller-647584dd76-mnzhh\" (UID: \"a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24\") " pod="opendatahub/maas-controller-647584dd76-mnzhh"
May 28 15:18:11.289011 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.288983    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fpqfk\" (UniqueName: \"kubernetes.io/projected/a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24-kube-api-access-fpqfk\") pod \"maas-controller-647584dd76-mnzhh\" (UID: \"a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24\") " pod="opendatahub/maas-controller-647584dd76-mnzhh"
May 28 15:18:11.296959 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.296932    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpqfk\" (UniqueName: \"kubernetes.io/projected/a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24-kube-api-access-fpqfk\") pod \"maas-controller-647584dd76-mnzhh\" (UID: \"a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24\") " pod="opendatahub/maas-controller-647584dd76-mnzhh"
May 28 15:18:11.478440 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.478359    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-647584dd76-mnzhh"
May 28 15:18:11.606796 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.606770    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-647584dd76-mnzhh"]
May 28 15:18:11.608502 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:18:11.608461    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c4a4bd_2cd6_48eb_957f_8fcf853c7b24.slice/crio-97d76e98736ccd95cc5dbba6f93321f5db49fbce18bd44c613acf6a96afb0e1e WatchSource:0}: Error finding container 97d76e98736ccd95cc5dbba6f93321f5db49fbce18bd44c613acf6a96afb0e1e: Status 404 returned error can't find the container with id 97d76e98736ccd95cc5dbba6f93321f5db49fbce18bd44c613acf6a96afb0e1e
May 28 15:18:11.742016 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:11.741934    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-647584dd76-mnzhh" event={"ID":"a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24","Type":"ContainerStarted","Data":"97d76e98736ccd95cc5dbba6f93321f5db49fbce18bd44c613acf6a96afb0e1e"}
May 28 15:18:14.754659 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:14.754621    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-647584dd76-mnzhh" event={"ID":"a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24","Type":"ContainerStarted","Data":"bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786"}
May 28 15:18:14.755089 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:14.754685    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-647584dd76-mnzhh"
May 28 15:18:14.772211 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:14.772161    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-647584dd76-mnzhh" podStartSLOduration=1.365778808 podStartE2EDuration="3.772149332s" podCreationTimestamp="2026-05-28 15:18:11 +0000 UTC" firstStartedPulling="2026-05-28 15:18:11.609617863 +0000 UTC m=+628.394768501" lastFinishedPulling="2026-05-28 15:18:14.015988365 +0000 UTC m=+630.801139025" observedRunningTime="2026-05-28 15:18:14.770084356 +0000 UTC m=+631.555235017" watchObservedRunningTime="2026-05-28 15:18:14.772149332 +0000 UTC m=+631.557299992"
May 28 15:18:25.767669 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:25.767632    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-647584dd76-mnzhh"
May 28 15:18:59.532833 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.532795    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"]
May 28 15:18:59.535751 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.535730    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.539764 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.539738    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 28 15:18:59.539764 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.539756    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jts6z\""
May 28 15:18:59.539944 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.539769    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
May 28 15:18:59.539944 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.539848    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 28 15:18:59.545086 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.545065    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"]
May 28 15:18:59.677313 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.677284    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.677475 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.677322    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.677475 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.677350    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.677475 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.677420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgrdc\" (UniqueName: \"kubernetes.io/projected/025138f4-ba4f-4bf6-acec-599c216c918e-kube-api-access-xgrdc\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.677475 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.677452    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.677663 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.677482    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/025138f4-ba4f-4bf6-acec-599c216c918e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.778683 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.778645    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.778862 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.778699    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.778862 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.778741    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.778862 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.778782    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xgrdc\" (UniqueName: \"kubernetes.io/projected/025138f4-ba4f-4bf6-acec-599c216c918e-kube-api-access-xgrdc\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.778862 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.778817    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.778862 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.778847    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/025138f4-ba4f-4bf6-acec-599c216c918e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.779124 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.779081    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.779192 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.779136    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.779192 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.779151    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.781205 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.781184    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/025138f4-ba4f-4bf6-acec-599c216c918e-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.781529 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.781510    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/025138f4-ba4f-4bf6-acec-599c216c918e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.787812 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.787755    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgrdc\" (UniqueName: \"kubernetes.io/projected/025138f4-ba4f-4bf6-acec-599c216c918e-kube-api-access-xgrdc\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b\" (UID: \"025138f4-ba4f-4bf6-acec-599c216c918e\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.846513 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.846479    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:18:59.987622 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.987595    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"]
May 28 15:18:59.989289 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:18:59.989256    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025138f4_ba4f_4bf6_acec_599c216c918e.slice/crio-dfbed239af3b3d1b3f9c9f4e8941d1009f7ade1f6d05cece8184e6615a1552c9 WatchSource:0}: Error finding container dfbed239af3b3d1b3f9c9f4e8941d1009f7ade1f6d05cece8184e6615a1552c9: Status 404 returned error can't find the container with id dfbed239af3b3d1b3f9c9f4e8941d1009f7ade1f6d05cece8184e6615a1552c9
May 28 15:18:59.991101 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:18:59.991081    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 15:19:00.911945 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:00.911898    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b" event={"ID":"025138f4-ba4f-4bf6-acec-599c216c918e","Type":"ContainerStarted","Data":"dfbed239af3b3d1b3f9c9f4e8941d1009f7ade1f6d05cece8184e6615a1552c9"}
May 28 15:19:07.938303 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:07.938269    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b" event={"ID":"025138f4-ba4f-4bf6-acec-599c216c918e","Type":"ContainerStarted","Data":"07135a447efaa42938a6f723d96bf77bb5b92efc3fd6cc5ad132a8637a196a99"}
May 28 15:19:11.507528 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.507491    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"]
May 28 15:19:11.510960 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.510939    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.513731 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.513705    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
May 28 15:19:11.520817 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.520787    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"]
May 28 15:19:11.569615 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.569573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.569776 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.569623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.569776 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.569669    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.569776 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.569701    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-469kx\" (UniqueName: \"kubernetes.io/projected/09be5615-6e8f-4b70-8a47-cc0a72bb13de-kube-api-access-469kx\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.569776 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.569740    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09be5615-6e8f-4b70-8a47-cc0a72bb13de-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.569776 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.569766    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.670947 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.670900    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.671124 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.670959    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.671124 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.671015    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.671124 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.671052    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-469kx\" (UniqueName: \"kubernetes.io/projected/09be5615-6e8f-4b70-8a47-cc0a72bb13de-kube-api-access-469kx\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.671124 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.671099    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09be5615-6e8f-4b70-8a47-cc0a72bb13de-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.671353 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.671130    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.671521 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.671500    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.671945 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.671886    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.672260 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.672239    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.674237 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.674210    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09be5615-6e8f-4b70-8a47-cc0a72bb13de-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.674738 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.674720    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09be5615-6e8f-4b70-8a47-cc0a72bb13de-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.684921 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.684890    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-469kx\" (UniqueName: \"kubernetes.io/projected/09be5615-6e8f-4b70-8a47-cc0a72bb13de-kube-api-access-469kx\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7\" (UID: \"09be5615-6e8f-4b70-8a47-cc0a72bb13de\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.822506 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.822477    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:11.972228 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:11.968958    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"]
May 28 15:19:11.976465 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:19:11.976438    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09be5615_6e8f_4b70_8a47_cc0a72bb13de.slice/crio-a2ca99c5ddfd8befcdb49c7c4fe81b5bfb93a4374e494243c29809e126ce76f5 WatchSource:0}: Error finding container a2ca99c5ddfd8befcdb49c7c4fe81b5bfb93a4374e494243c29809e126ce76f5: Status 404 returned error can't find the container with id a2ca99c5ddfd8befcdb49c7c4fe81b5bfb93a4374e494243c29809e126ce76f5
May 28 15:19:12.963058 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:12.963019    2560 generic.go:358] "Generic (PLEG): container finished" podID="025138f4-ba4f-4bf6-acec-599c216c918e" containerID="07135a447efaa42938a6f723d96bf77bb5b92efc3fd6cc5ad132a8637a196a99" exitCode=0
May 28 15:19:12.963433 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:12.963090    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b" event={"ID":"025138f4-ba4f-4bf6-acec-599c216c918e","Type":"ContainerDied","Data":"07135a447efaa42938a6f723d96bf77bb5b92efc3fd6cc5ad132a8637a196a99"}
May 28 15:19:12.964797 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:12.964773    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7" event={"ID":"09be5615-6e8f-4b70-8a47-cc0a72bb13de","Type":"ContainerStarted","Data":"2333e590dc51244483fc12bf0e70e698c5d317fe1d75001a8dc272a81e07f091"}
May 28 15:19:12.964886 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:12.964804    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7" event={"ID":"09be5615-6e8f-4b70-8a47-cc0a72bb13de","Type":"ContainerStarted","Data":"a2ca99c5ddfd8befcdb49c7c4fe81b5bfb93a4374e494243c29809e126ce76f5"}
May 28 15:19:14.977510 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:14.977470    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b" event={"ID":"025138f4-ba4f-4bf6-acec-599c216c918e","Type":"ContainerStarted","Data":"b970b7460d18a9943de11adbb83fe0418d7e79afbb4c993f283618f07458ce77"}
May 28 15:19:14.977985 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:14.977771    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:19:14.998865 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:14.998817    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b" podStartSLOduration=1.485702761 podStartE2EDuration="15.998803972s" podCreationTimestamp="2026-05-28 15:18:59 +0000 UTC" firstStartedPulling="2026-05-28 15:18:59.991256289 +0000 UTC m=+676.776406935" lastFinishedPulling="2026-05-28 15:19:14.504357507 +0000 UTC m=+691.289508146" observedRunningTime="2026-05-28 15:19:14.996754933 +0000 UTC m=+691.781905595" watchObservedRunningTime="2026-05-28 15:19:14.998803972 +0000 UTC m=+691.783954633"
May 28 15:19:17.990311 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:17.990218    2560 generic.go:358] "Generic (PLEG): container finished" podID="09be5615-6e8f-4b70-8a47-cc0a72bb13de" containerID="2333e590dc51244483fc12bf0e70e698c5d317fe1d75001a8dc272a81e07f091" exitCode=0
May 28 15:19:17.990311 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:17.990292    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7" event={"ID":"09be5615-6e8f-4b70-8a47-cc0a72bb13de","Type":"ContainerDied","Data":"2333e590dc51244483fc12bf0e70e698c5d317fe1d75001a8dc272a81e07f091"}
May 28 15:19:19.412438 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.412389    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"]
May 28 15:19:19.416451 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.416431    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.419304 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.419283    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
May 28 15:19:19.427388 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.427364    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"]
May 28 15:19:19.433038 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.433016    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b80caaba-4ba7-4bda-a236-5ac2280457dc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.433151 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.433054    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ln82\" (UniqueName: \"kubernetes.io/projected/b80caaba-4ba7-4bda-a236-5ac2280457dc-kube-api-access-7ln82\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.433194 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.433143    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.433240 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.433188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.433290 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.433272    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.433336 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.433322    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.533790 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.533757    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b80caaba-4ba7-4bda-a236-5ac2280457dc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.533790 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.533798    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7ln82\" (UniqueName: \"kubernetes.io/projected/b80caaba-4ba7-4bda-a236-5ac2280457dc-kube-api-access-7ln82\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.534036 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.533835    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.534036 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.533852    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.534036 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.533881    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.534036 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.533903    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.534344 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.534312    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.534456 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.534348    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.534456 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.534388    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.536186 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.536169    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b80caaba-4ba7-4bda-a236-5ac2280457dc-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.536501 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.536484    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b80caaba-4ba7-4bda-a236-5ac2280457dc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.542330 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.542297    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ln82\" (UniqueName: \"kubernetes.io/projected/b80caaba-4ba7-4bda-a236-5ac2280457dc-kube-api-access-7ln82\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l\" (UID: \"b80caaba-4ba7-4bda-a236-5ac2280457dc\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.727986 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.727910    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:19.864735 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.864701    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"]
May 28 15:19:19.999843 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.999753    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l" event={"ID":"b80caaba-4ba7-4bda-a236-5ac2280457dc","Type":"ContainerStarted","Data":"e00c619c3e5584604287337886fa7a1e9754dfed251abc2739f1769b06f9267a"}
May 28 15:19:19.999843 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:19.999801    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l" event={"ID":"b80caaba-4ba7-4bda-a236-5ac2280457dc","Type":"ContainerStarted","Data":"da441441dfa5249fed00cb182bceb33204360565339c12f06676a105f46872cd"}
May 28 15:19:22.009186 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:22.009086    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7" event={"ID":"09be5615-6e8f-4b70-8a47-cc0a72bb13de","Type":"ContainerStarted","Data":"863bd72ff9a74a1ae69e5b4f382aabfed42e3b46be692afc052f6b537fb161cb"}
May 28 15:19:22.009592 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:22.009384    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:22.027489 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:22.027435    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7" podStartSLOduration=7.417554952 podStartE2EDuration="11.027422124s" podCreationTimestamp="2026-05-28 15:19:11 +0000 UTC" firstStartedPulling="2026-05-28 15:19:17.990926926 +0000 UTC m=+694.776077563" lastFinishedPulling="2026-05-28 15:19:21.600794097 +0000 UTC m=+698.385944735" observedRunningTime="2026-05-28 15:19:22.025368653 +0000 UTC m=+698.810519314" watchObservedRunningTime="2026-05-28 15:19:22.027422124 +0000 UTC m=+698.812572784"
May 28 15:19:25.995774 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:25.995735    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b"
May 28 15:19:26.024339 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:26.024309    2560 generic.go:358] "Generic (PLEG): container finished" podID="b80caaba-4ba7-4bda-a236-5ac2280457dc" containerID="e00c619c3e5584604287337886fa7a1e9754dfed251abc2739f1769b06f9267a" exitCode=0
May 28 15:19:26.024339 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:26.024337    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l" event={"ID":"b80caaba-4ba7-4bda-a236-5ac2280457dc","Type":"ContainerDied","Data":"e00c619c3e5584604287337886fa7a1e9754dfed251abc2739f1769b06f9267a"}
May 28 15:19:27.030287 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:27.030252    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l" event={"ID":"b80caaba-4ba7-4bda-a236-5ac2280457dc","Type":"ContainerStarted","Data":"42213fdda5f7ab91609c818284d8595db3c0094356edaebfa5fd14ca57718f0c"}
May 28 15:19:27.030692 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:27.030468    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:27.050312 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:27.050266    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l" podStartSLOduration=7.563772477 podStartE2EDuration="8.050252148s" podCreationTimestamp="2026-05-28 15:19:19 +0000 UTC" firstStartedPulling="2026-05-28 15:19:26.024969412 +0000 UTC m=+702.810120050" lastFinishedPulling="2026-05-28 15:19:26.511449079 +0000 UTC m=+703.296599721" observedRunningTime="2026-05-28 15:19:27.048866057 +0000 UTC m=+703.834016718" watchObservedRunningTime="2026-05-28 15:19:27.050252148 +0000 UTC m=+703.835402808"
May 28 15:19:30.917081 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:30.917041    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"]
May 28 15:19:30.933295 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:30.933262    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"]
May 28 15:19:30.933498 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:30.933405    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:30.936418 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:30.936395    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
May 28 15:19:31.039861 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.039833    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.040025 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.039882    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng8db\" (UniqueName: \"kubernetes.io/projected/98b08613-12ce-46c2-a658-09657dfdc263-kube-api-access-ng8db\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.040025 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.039931    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.040025 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.039998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b08613-12ce-46c2-a658-09657dfdc263-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.040131 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.040039    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.040131 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.040058    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.140674 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.140640    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b08613-12ce-46c2-a658-09657dfdc263-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.140847 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.140690    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.140847 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.140723    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.140847 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.140747    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.140847 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.140777    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ng8db\" (UniqueName: \"kubernetes.io/projected/98b08613-12ce-46c2-a658-09657dfdc263-kube-api-access-ng8db\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.141018 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.140893    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.141225 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.141197    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.141393 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.141236    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.141393 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.141311    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.143335 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.143313    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b08613-12ce-46c2-a658-09657dfdc263-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.143438 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.143355    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b08613-12ce-46c2-a658-09657dfdc263-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.148417 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.148397    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng8db\" (UniqueName: \"kubernetes.io/projected/98b08613-12ce-46c2-a658-09657dfdc263-kube-api-access-ng8db\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf\" (UID: \"98b08613-12ce-46c2-a658-09657dfdc263\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.243422 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.243339    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:31.381504 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:31.381482    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"]
May 28 15:19:31.383199 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:19:31.383168    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b08613_12ce_46c2_a658_09657dfdc263.slice/crio-4a9f22efb44fd652b958d3737dc2d9d1dd4e42a61e805e320ce3851f0cae5e8b WatchSource:0}: Error finding container 4a9f22efb44fd652b958d3737dc2d9d1dd4e42a61e805e320ce3851f0cae5e8b: Status 404 returned error can't find the container with id 4a9f22efb44fd652b958d3737dc2d9d1dd4e42a61e805e320ce3851f0cae5e8b
May 28 15:19:32.051316 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:32.051281    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf" event={"ID":"98b08613-12ce-46c2-a658-09657dfdc263","Type":"ContainerStarted","Data":"c21c46177bcbdd9ae1beb34955891a389b4d8279e570275d39eea4f4f10381c9"}
May 28 15:19:32.051316 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:32.051318    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf" event={"ID":"98b08613-12ce-46c2-a658-09657dfdc263","Type":"ContainerStarted","Data":"4a9f22efb44fd652b958d3737dc2d9d1dd4e42a61e805e320ce3851f0cae5e8b"}
May 28 15:19:33.025585 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:33.025538    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7"
May 28 15:19:38.047007 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:38.046969    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l"
May 28 15:19:41.092885 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:41.092850    2560 generic.go:358] "Generic (PLEG): container finished" podID="98b08613-12ce-46c2-a658-09657dfdc263" containerID="c21c46177bcbdd9ae1beb34955891a389b4d8279e570275d39eea4f4f10381c9" exitCode=0
May 28 15:19:41.093248 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:41.092899    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf" event={"ID":"98b08613-12ce-46c2-a658-09657dfdc263","Type":"ContainerDied","Data":"c21c46177bcbdd9ae1beb34955891a389b4d8279e570275d39eea4f4f10381c9"}
May 28 15:19:42.097485 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:42.097446    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf" event={"ID":"98b08613-12ce-46c2-a658-09657dfdc263","Type":"ContainerStarted","Data":"a985a9c7d25dc9ecdbb74d76915440d95f8fe80625ee9fbda0564e1221ebcaa7"}
May 28 15:19:42.097882 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:42.097656    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:19:42.119477 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:42.119432    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf" podStartSLOduration=11.803981637 podStartE2EDuration="12.119418784s" podCreationTimestamp="2026-05-28 15:19:30 +0000 UTC" firstStartedPulling="2026-05-28 15:19:41.093475862 +0000 UTC m=+717.878626500" lastFinishedPulling="2026-05-28 15:19:41.408913008 +0000 UTC m=+718.194063647" observedRunningTime="2026-05-28 15:19:42.117413755 +0000 UTC m=+718.902564639" watchObservedRunningTime="2026-05-28 15:19:42.119418784 +0000 UTC m=+718.904569443"
May 28 15:19:53.113889 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:19:53.113862    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf"
May 28 15:21:51.057651 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.057615    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-647584dd76-mnzhh"]
May 28 15:21:51.058316 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.058286    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-647584dd76-mnzhh" podUID="a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24" containerName="manager" containerID="cri-o://bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786" gracePeriod=30
May 28 15:21:51.301569 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.301532    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-647584dd76-mnzhh"
May 28 15:21:51.340688 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.340616    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpqfk\" (UniqueName: \"kubernetes.io/projected/a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24-kube-api-access-fpqfk\") pod \"a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24\" (UID: \"a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24\") "
May 28 15:21:51.342810 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.342775    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24-kube-api-access-fpqfk" (OuterVolumeSpecName: "kube-api-access-fpqfk") pod "a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24" (UID: "a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24"). InnerVolumeSpecName "kube-api-access-fpqfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 15:21:51.442091 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.442058    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpqfk\" (UniqueName: \"kubernetes.io/projected/a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24-kube-api-access-fpqfk\") on node \"ip-10-0-135-61.ec2.internal\" DevicePath \"\""
May 28 15:21:51.572342 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.572303    2560 generic.go:358] "Generic (PLEG): container finished" podID="a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24" containerID="bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786" exitCode=0
May 28 15:21:51.572498 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.572369    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-647584dd76-mnzhh"
May 28 15:21:51.572498 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.572386    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-647584dd76-mnzhh" event={"ID":"a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24","Type":"ContainerDied","Data":"bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786"}
May 28 15:21:51.572498 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.572429    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-647584dd76-mnzhh" event={"ID":"a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24","Type":"ContainerDied","Data":"97d76e98736ccd95cc5dbba6f93321f5db49fbce18bd44c613acf6a96afb0e1e"}
May 28 15:21:51.572498 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.572445    2560 scope.go:117] "RemoveContainer" containerID="bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786"
May 28 15:21:51.581982 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.581962    2560 scope.go:117] "RemoveContainer" containerID="bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786"
May 28 15:21:51.582240 ip-10-0-135-61 kubenswrapper[2560]: E0528 15:21:51.582220    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786\": container with ID starting with bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786 not found: ID does not exist" containerID="bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786"
May 28 15:21:51.582284 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.582248    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786"} err="failed to get container status \"bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786\": rpc error: code = NotFound desc = could not find container \"bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786\": container with ID starting with bfcfda858830fc2e46f0be2762b2837be1f5209a761c664a19de2fb5abf20786 not found: ID does not exist"
May 28 15:21:51.598154 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.598101    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-647584dd76-mnzhh"]
May 28 15:21:51.602212 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.602193    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-647584dd76-mnzhh"]
May 28 15:21:51.785619 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:21:51.785592    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24" path="/var/lib/kubelet/pods/a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24/volumes"
May 28 15:22:43.755458 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:22:43.755430    2560 scope.go:117] "RemoveContainer" containerID="b5a93473097092318bb5cfac33d3abb1e784a1e42b933c7642749fae139f3716"
May 28 15:42:56.686498 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:42:56.686464    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-8vgmq_c73378a5-13ea-4643-aa94-0f2f0fe08550/manager/0.log"
May 28 15:42:57.154707 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:42:57.154678    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-ll6lg_c36eea84-6483-4c02-b81e-9d95cf259cf0/manager/1.log"
May 28 15:42:57.389132 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:42:57.389104    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c44ff6d8-gdmjw_a0362333-2da4-44c2-ac26-30ffd0564bf3/manager/0.log"
May 28 15:42:58.735996 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:42:58.735964    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb_b6f38cb4-fab9-4e4b-b700-8562a6e74b35/util/0.log"
May 28 15:42:58.742048 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:42:58.742022    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb_b6f38cb4-fab9-4e4b-b700-8562a6e74b35/pull/0.log"
May 28 15:42:58.747978 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:42:58.747957    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb_b6f38cb4-fab9-4e4b-b700-8562a6e74b35/extract/0.log"
May 28 15:42:58.974782 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:42:58.974750    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-bzpbv_97ecd4c9-d5e2-4003-9a28-de1f09fe7b01/manager/0.log"
May 28 15:42:59.656400 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:42:59.656369    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-pbmsx_cd0c9b04-1571-4705-b2f7-a339b2165519/manager/0.log"
May 28 15:43:00.105072 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:00.105045    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-b9lw5_e211b9a4-02f3-4323-beb7-d2e06737ba30/discovery/0.log"
May 28 15:43:00.215913 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:00.215874    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b6697f59-48n6d_5690bd1c-ecce-4cc1-a862-6828e861dff2/kube-auth-proxy/0.log"
May 28 15:43:01.034083 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:01.034052    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b_025138f4-ba4f-4bf6-acec-599c216c918e/storage-initializer/0.log"
May 28 15:43:01.041318 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:01.041296    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-8tf5b_025138f4-ba4f-4bf6-acec-599c216c918e/main/0.log"
May 28 15:43:01.156684 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:01.156645    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7_09be5615-6e8f-4b70-8a47-cc0a72bb13de/storage-initializer/0.log"
May 28 15:43:01.163998 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:01.163981    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-72qk7_09be5615-6e8f-4b70-8a47-cc0a72bb13de/main/0.log"
May 28 15:43:01.504089 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:01.504061    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l_b80caaba-4ba7-4bda-a236-5ac2280457dc/storage-initializer/0.log"
May 28 15:43:01.512065 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:01.512026    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-t5k9l_b80caaba-4ba7-4bda-a236-5ac2280457dc/main/0.log"
May 28 15:43:01.623540 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:01.623517    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf_98b08613-12ce-46c2-a658-09657dfdc263/storage-initializer/0.log"
May 28 15:43:01.629578 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:01.629540    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-v8tvf_98b08613-12ce-46c2-a658-09657dfdc263/main/0.log"
May 28 15:43:08.574808 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:08.574735    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sdbjv_f37780de-3086-4fb5-898e-ed20851982b7/global-pull-secret-syncer/0.log"
May 28 15:43:08.684150 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:08.684123    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w6hk4_5ecd7b4a-2ba1-4753-ba17-959add9303eb/konnectivity-agent/0.log"
May 28 15:43:08.702520 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:08.702494    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-61.ec2.internal_a3d58cbef8327a74cc29c171ea36677d/haproxy/0.log"
May 28 15:43:12.406782 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:12.406753    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb_b6f38cb4-fab9-4e4b-b700-8562a6e74b35/extract/0.log"
May 28 15:43:12.433792 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:12.433758    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb_b6f38cb4-fab9-4e4b-b700-8562a6e74b35/util/0.log"
May 28 15:43:12.460591 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:12.460536    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hwrgb_b6f38cb4-fab9-4e4b-b700-8562a6e74b35/pull/0.log"
May 28 15:43:12.666653 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:12.666572    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-bzpbv_97ecd4c9-d5e2-4003-9a28-de1f09fe7b01/manager/0.log"
May 28 15:43:12.929432 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:12.929342    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-pbmsx_cd0c9b04-1571-4705-b2f7-a339b2165519/manager/0.log"
May 28 15:43:14.488134 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:14.488104    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-2pr9r_4e22aae7-e299-4b3e-b9ff-03811e253020/monitoring-plugin/0.log"
May 28 15:43:14.682146 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:14.682063    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-grrmk_b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7/node-exporter/0.log"
May 28 15:43:14.706009 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:14.705983    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-grrmk_b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7/kube-rbac-proxy/0.log"
May 28 15:43:14.729598 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:14.729575    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-grrmk_b929aa2f-d9d3-42e4-a0bd-44d6e5b90ae7/init-textfile/0.log"
May 28 15:43:14.763542 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:14.763519    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-cv4tn_a059a4f0-1102-4713-a599-251fefeb0bd8/kube-rbac-proxy-main/0.log"
May 28 15:43:14.782876 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:14.782859    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-cv4tn_a059a4f0-1102-4713-a599-251fefeb0bd8/kube-rbac-proxy-self/0.log"
May 28 15:43:14.807109 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:14.807083    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-cv4tn_a059a4f0-1102-4713-a599-251fefeb0bd8/openshift-state-metrics/0.log"
May 28 15:43:16.901677 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:16.901642    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"]
May 28 15:43:16.902030 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:16.901962    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24" containerName="manager"
May 28 15:43:16.902030 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:16.901973    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24" containerName="manager"
May 28 15:43:16.902099 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:16.902037    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="a2c4a4bd-2cd6-48eb-957f-8fcf853c7b24" containerName="manager"
May 28 15:43:16.909298 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:16.909268    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:16.911368 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:16.911342    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"]
May 28 15:43:16.911914 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:16.911887    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkl9v\"/\"kube-root-ca.crt\""
May 28 15:43:16.912015 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:16.911951    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xkl9v\"/\"default-dockercfg-vtnc5\""
May 28 15:43:16.912077 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:16.911895    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkl9v\"/\"openshift-service-ca.crt\""
May 28 15:43:17.033168 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.033133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-lib-modules\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.033338 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.033192    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgc28\" (UniqueName: \"kubernetes.io/projected/1c9d7a38-802f-41c0-80a5-bde33cca2422-kube-api-access-fgc28\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.033338 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.033289    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-proc\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.033338 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.033325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-podres\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.033476 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.033349    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-sys\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.134015 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.133981    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fgc28\" (UniqueName: \"kubernetes.io/projected/1c9d7a38-802f-41c0-80a5-bde33cca2422-kube-api-access-fgc28\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.134200 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.134032    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-proc\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.134200 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.134050    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-podres\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.134200 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.134129    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-proc\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.134200 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.134145    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-podres\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.134200 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.134160    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-sys\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.134409 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.134214    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-lib-modules\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.134409 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.134265    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-sys\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.134409 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.134347    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c9d7a38-802f-41c0-80a5-bde33cca2422-lib-modules\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.146588 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.146537    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgc28\" (UniqueName: \"kubernetes.io/projected/1c9d7a38-802f-41c0-80a5-bde33cca2422-kube-api-access-fgc28\") pod \"perf-node-gather-daemonset-kd4v7\" (UID: \"1c9d7a38-802f-41c0-80a5-bde33cca2422\") " pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.219925 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.219859    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:17.349755 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.349730    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"]
May 28 15:43:17.351939 ip-10-0-135-61 kubenswrapper[2560]: W0528 15:43:17.351908    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1c9d7a38_802f_41c0_80a5_bde33cca2422.slice/crio-968ca5e5b2b110189bb41c640ee9c85e4dfd304b7d5ca844dc0b6cab61d2c1d4 WatchSource:0}: Error finding container 968ca5e5b2b110189bb41c640ee9c85e4dfd304b7d5ca844dc0b6cab61d2c1d4: Status 404 returned error can't find the container with id 968ca5e5b2b110189bb41c640ee9c85e4dfd304b7d5ca844dc0b6cab61d2c1d4
May 28 15:43:17.353444 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:17.353427    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 15:43:18.138877 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:18.138834    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7" event={"ID":"1c9d7a38-802f-41c0-80a5-bde33cca2422","Type":"ContainerStarted","Data":"cf5580330124b8ea188b09be10f586fdea05359241b22e296d9040ade3342aae"}
May 28 15:43:18.138877 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:18.138877    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7" event={"ID":"1c9d7a38-802f-41c0-80a5-bde33cca2422","Type":"ContainerStarted","Data":"968ca5e5b2b110189bb41c640ee9c85e4dfd304b7d5ca844dc0b6cab61d2c1d4"}
May 28 15:43:18.139319 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:18.138928    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:18.156609 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:18.156566    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7" podStartSLOduration=2.156538425 podStartE2EDuration="2.156538425s" podCreationTimestamp="2026-05-28 15:43:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:43:18.154006301 +0000 UTC m=+2134.939156962" watchObservedRunningTime="2026-05-28 15:43:18.156538425 +0000 UTC m=+2134.941689084"
May 28 15:43:18.856427 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:18.856401    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vs6bq_cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902/dns/0.log"
May 28 15:43:18.875150 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:18.875126    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vs6bq_cef8aaac-b3b6-4a7a-bc07-6c40d2f2b902/kube-rbac-proxy/0.log"
May 28 15:43:18.936619 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:18.936593    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wgnqk_8f283641-0611-4d23-b85e-28cc5090beb1/dns-node-resolver/0.log"
May 28 15:43:19.437752 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:19.437724    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g6dcp_757cacce-776f-4504-94fe-de28576cf4b4/node-ca/0.log"
May 28 15:43:20.279909 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:20.279863    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-b9lw5_e211b9a4-02f3-4323-beb7-d2e06737ba30/discovery/0.log"
May 28 15:43:20.300753 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:20.300728    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b6697f59-48n6d_5690bd1c-ecce-4cc1-a862-6828e861dff2/kube-auth-proxy/0.log"
May 28 15:43:20.979528 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:20.979503    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nwwjb_ae300cc7-3fd0-4422-816d-eec9479c9a0b/serve-healthcheck-canary/0.log"
May 28 15:43:21.473863 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:21.473826    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8rkr6_30ab8040-39a2-48a1-8dbd-f543a30b379d/kube-rbac-proxy/0.log"
May 28 15:43:21.493048 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:21.493023    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8rkr6_30ab8040-39a2-48a1-8dbd-f543a30b379d/exporter/0.log"
May 28 15:43:21.512462 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:21.512437    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8rkr6_30ab8040-39a2-48a1-8dbd-f543a30b379d/extractor/0.log"
May 28 15:43:23.315936 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:23.315903    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-8vgmq_c73378a5-13ea-4643-aa94-0f2f0fe08550/manager/0.log"
May 28 15:43:23.465968 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:23.465938    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-ll6lg_c36eea84-6483-4c02-b81e-9d95cf259cf0/manager/0.log"
May 28 15:43:23.477511 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:23.477489    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-ll6lg_c36eea84-6483-4c02-b81e-9d95cf259cf0/manager/1.log"
May 28 15:43:23.555159 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:23.555135    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c44ff6d8-gdmjw_a0362333-2da4-44c2-ac26-30ffd0564bf3/manager/0.log"
May 28 15:43:24.152997 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:24.152968    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xkl9v/perf-node-gather-daemonset-kd4v7"
May 28 15:43:30.059453 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.059424    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4bxkw_1aea7a9e-2599-4468-9bec-d293c48f84a9/kube-multus/0.log"
May 28 15:43:30.366298 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.366226    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8nt5_30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc/kube-multus-additional-cni-plugins/0.log"
May 28 15:43:30.389448 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.389422    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8nt5_30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc/egress-router-binary-copy/0.log"
May 28 15:43:30.408912 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.408886    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8nt5_30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc/cni-plugins/0.log"
May 28 15:43:30.428540 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.428516    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8nt5_30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc/bond-cni-plugin/0.log"
May 28 15:43:30.447226 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.447207    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8nt5_30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc/routeoverride-cni/0.log"
May 28 15:43:30.466679 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.466661    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8nt5_30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc/whereabouts-cni-bincopy/0.log"
May 28 15:43:30.486621 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.486599    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8nt5_30d8e3e3-d6e7-4f9d-a050-a302d06bf8bc/whereabouts-cni/0.log"
May 28 15:43:30.578211 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.578184    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8jlgg_5086afac-ec94-48f4-8180-672970697414/network-metrics-daemon/0.log"
May 28 15:43:30.599439 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:30.599421    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8jlgg_5086afac-ec94-48f4-8180-672970697414/kube-rbac-proxy/0.log"
May 28 15:43:32.025064 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:32.025036    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2r4x_5344fe70-83db-47a2-bf7f-fe4b0659ba27/ovn-controller/0.log"
May 28 15:43:32.050763 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:32.050737    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2r4x_5344fe70-83db-47a2-bf7f-fe4b0659ba27/ovn-acl-logging/0.log"
May 28 15:43:32.068321 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:32.068294    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2r4x_5344fe70-83db-47a2-bf7f-fe4b0659ba27/kube-rbac-proxy-node/0.log"
May 28 15:43:32.086911 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:32.086887    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2r4x_5344fe70-83db-47a2-bf7f-fe4b0659ba27/kube-rbac-proxy-ovn-metrics/0.log"
May 28 15:43:32.105327 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:32.105307    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2r4x_5344fe70-83db-47a2-bf7f-fe4b0659ba27/northd/0.log"
May 28 15:43:32.123859 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:32.123832    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2r4x_5344fe70-83db-47a2-bf7f-fe4b0659ba27/nbdb/0.log"
May 28 15:43:32.143519 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:32.143500    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2r4x_5344fe70-83db-47a2-bf7f-fe4b0659ba27/sbdb/0.log"
May 28 15:43:32.236626 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:32.236539    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2r4x_5344fe70-83db-47a2-bf7f-fe4b0659ba27/ovnkube-controller/0.log"
May 28 15:43:33.310630 ip-10-0-135-61 kubenswrapper[2560]: I0528 15:43:33.310603    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-rmtwg_237da9f1-53ff-4878-9254-e0a1cc9a01db/network-check-target-container/0.log"