Jun 02 02:45:11.255718 ip-10-0-129-101 systemd[1]: Starting Kubernetes Kubelet...
Jun 02 02:45:11.727139 ip-10-0-129-101 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 02:45:11.727139 ip-10-0-129-101 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jun 02 02:45:11.727139 ip-10-0-129-101 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 02:45:11.727139 ip-10-0-129-101 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jun 02 02:45:11.727139 ip-10-0-129-101 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 02:45:11.728823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.728689    2554 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jun 02 02:45:11.736000 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.735977    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 02:45:11.736000 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.735995    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 02:45:11.736000 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736000    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 02:45:11.736000 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736003    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 02:45:11.736000 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736006    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 02:45:11.736000 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736008    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736011    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736014    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736017    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736019    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736022    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736025    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736028    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736031    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736033    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736036    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736039    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736050    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736053    2554 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736056    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736058    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736061    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736064    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736066    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736069    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 02:45:11.736216 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736072    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736074    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736078    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736082    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736085    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736088    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736091    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736093    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736096    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736099    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736101    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736104    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736107    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736110    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736114    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736117    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736119    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736122    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736124    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 02:45:11.736756 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736127    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736131    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736133    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736137    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736139    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736142    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736146    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736149    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736151    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736154    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736156    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736159    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736161    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736164    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736166    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736169    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736171    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736175    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736179    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 02:45:11.737315 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736184    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736187    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736190    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736192    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736195    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736198    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736200    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736203    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736207    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736211    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736213    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736216    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736219    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736222    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736225    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736229    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736231    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736234    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736237    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736239    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 02:45:11.737808 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736242    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736244    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.736247    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737252    2554 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737259    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737263    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737266    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737268    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737271    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737274    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737276    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737279    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737282    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737285    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737288    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737290    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737292    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737295    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737298    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737302    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 02:45:11.738299 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737306    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737309    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737312    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737315    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737318    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737321    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737323    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737327    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737330    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737332    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737335    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737337    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737340    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737342    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737345    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737347    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737350    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737352    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737355    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737357    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 02:45:11.738798 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737360    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737362    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737365    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737368    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737371    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737374    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737377    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737379    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737381    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737384    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737386    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737389    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737392    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737394    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737397    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737399    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737402    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737405    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737407    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737410    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 02:45:11.739331 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737413    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737415    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737419    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737422    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737425    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737427    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737430    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737432    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737434    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737437    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737440    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737443    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737445    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737448    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737466    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737469    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737472    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737475    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737477    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 02:45:11.739830 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737480    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737483    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737485    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737488    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737490    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737493    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737496    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737499    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737501    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.737504    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737573    2554 flags.go:64] FLAG: --address="0.0.0.0"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737581    2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737587    2554 flags.go:64] FLAG: --anonymous-auth="true"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737592    2554 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737597    2554 flags.go:64] FLAG: --authentication-token-webhook="false"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737600    2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737604    2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737609    2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737612    2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737615    2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737619    2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jun 02 02:45:11.740303 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737622    2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737625    2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737628    2554 flags.go:64] FLAG: --cgroup-root=""
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737631    2554 flags.go:64] FLAG: --cgroups-per-qos="true"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737634    2554 flags.go:64] FLAG: --client-ca-file=""
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737637    2554 flags.go:64] FLAG: --cloud-config=""
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737639    2554 flags.go:64] FLAG: --cloud-provider="external"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737643    2554 flags.go:64] FLAG: --cluster-dns="[]"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737647    2554 flags.go:64] FLAG: --cluster-domain=""
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737650    2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737653    2554 flags.go:64] FLAG: --config-dir=""
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737656    2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737659    2554 flags.go:64] FLAG: --container-log-max-files="5"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737663    2554 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737666    2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737669    2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737675    2554 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737678    2554 flags.go:64] FLAG: --contention-profiling="false"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737681    2554 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737684    2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737687    2554 flags.go:64] FLAG: --cpu-manager-policy="none"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737690    2554 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737695    2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737698    2554 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737701    2554 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jun 02 02:45:11.740834 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737703    2554 flags.go:64] FLAG: --enable-load-reader="false"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737707    2554 flags.go:64] FLAG: --enable-server="true"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737710    2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737715    2554 flags.go:64] FLAG: --event-burst="100"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737718    2554 flags.go:64] FLAG: --event-qps="50"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737722    2554 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737725    2554 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737744    2554 flags.go:64] FLAG: --eviction-hard=""
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737748    2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737751    2554 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737754    2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737757    2554 flags.go:64] FLAG: --eviction-soft=""
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737760    2554 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737763    2554 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737766    2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737769    2554 flags.go:64] FLAG: --experimental-mounter-path=""
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737772    2554 flags.go:64] FLAG: --fail-cgroupv1="false"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737775    2554 flags.go:64] FLAG: --fail-swap-on="true"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737778    2554 flags.go:64] FLAG: --feature-gates=""
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737782    2554 flags.go:64] FLAG: --file-check-frequency="20s"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737785    2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737788    2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737791    2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737795    2554 flags.go:64] FLAG: --healthz-port="10248"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737798    2554 flags.go:64] FLAG: --help="false"
Jun 02 02:45:11.741430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737801    2554 flags.go:64] FLAG: --hostname-override="ip-10-0-129-101.ec2.internal"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737804    2554 flags.go:64] FLAG: --housekeeping-interval="10s"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737807    2554 flags.go:64] FLAG: --http-check-frequency="20s"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737810    2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737813    2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737817    2554 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737820    2554 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737823    2554 flags.go:64] FLAG: --image-service-endpoint=""
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737826    2554 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737829    2554 flags.go:64] FLAG: --kube-api-burst="100"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737833    2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737836    2554 flags.go:64] FLAG: --kube-api-qps="50"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737839    2554 flags.go:64] FLAG: --kube-reserved=""
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737842    2554 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737845    2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737848    2554 flags.go:64] FLAG: --kubelet-cgroups=""
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737851    2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737854    2554 flags.go:64] FLAG: --lock-file=""
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737857    2554 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737859    2554 flags.go:64] FLAG: --log-flush-frequency="5s"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737862    2554 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737868    2554 flags.go:64] FLAG: --log-json-split-stream="false"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737871    2554 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737874    2554 flags.go:64] FLAG: --log-text-split-stream="false"
Jun 02 02:45:11.742049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737877    2554 flags.go:64] FLAG: --logging-format="text"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737880    2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737883    2554 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737886    2554 flags.go:64] FLAG: --manifest-url=""
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737889    2554 flags.go:64] FLAG: --manifest-url-header=""
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737893    2554 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737897    2554 flags.go:64] FLAG: --max-open-files="1000000"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737901    2554 flags.go:64] FLAG: --max-pods="110"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737904    2554 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737907    2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737910    2554 flags.go:64] FLAG: --memory-manager-policy="None"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737913    2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737916    2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737919    2554 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737922    2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737929    2554 flags.go:64] FLAG: --node-status-max-images="50"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737932    2554 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737936    2554 flags.go:64] FLAG: --oom-score-adj="-999"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737939    2554 flags.go:64] FLAG: --pod-cidr=""
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737942    2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737947    2554 flags.go:64] FLAG: --pod-manifest-path=""
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737950    2554 flags.go:64] FLAG: --pod-max-pids="-1"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737953    2554 flags.go:64] FLAG: --pods-per-core="0"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737956    2554 flags.go:64] FLAG: --port="10250"
Jun 02 02:45:11.742692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737959    2554 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737962    2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04aa608ab5c35bba9"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737966    2554 flags.go:64] FLAG: --qos-reserved=""
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737969    2554 flags.go:64] FLAG: --read-only-port="10255"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737972    2554 flags.go:64] FLAG: --register-node="true"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737975    2554 flags.go:64] FLAG: --register-schedulable="true"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737977    2554 flags.go:64] FLAG: --register-with-taints=""
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737981    2554 flags.go:64] FLAG: --registry-burst="10"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737984    2554 flags.go:64] FLAG: --registry-qps="5"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737986    2554 flags.go:64] FLAG: --reserved-cpus=""
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737990    2554 flags.go:64] FLAG: --reserved-memory=""
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737993    2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737996    2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.737999    2554 flags.go:64] FLAG: --rotate-certificates="false"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738002    2554 flags.go:64] FLAG: --rotate-server-certificates="false"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738006    2554 flags.go:64] FLAG: --runonce="false"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738009    2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738012    2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738015    2554 flags.go:64] FLAG: --seccomp-default="false"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738018    2554 flags.go:64] FLAG: --serialize-image-pulls="true"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738021    2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738024    2554 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738027    2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738030    2554 flags.go:64] FLAG: --storage-driver-password="root"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738033    2554 flags.go:64] FLAG: --storage-driver-secure="false"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738035    2554 flags.go:64] FLAG: --storage-driver-table="stats"
Jun 02 02:45:11.743276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738038    2554 flags.go:64] FLAG: --storage-driver-user="root"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738041    2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738044    2554 flags.go:64] FLAG: --sync-frequency="1m0s"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738047    2554 flags.go:64] FLAG: --system-cgroups=""
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738050    2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738055    2554 flags.go:64] FLAG: --system-reserved-cgroup=""
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738058    2554 flags.go:64] FLAG: --tls-cert-file=""
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738061    2554 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738069    2554 flags.go:64] FLAG: --tls-min-version=""
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738072    2554 flags.go:64] FLAG: --tls-private-key-file=""
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738075    2554 flags.go:64] FLAG: --topology-manager-policy="none"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738078    2554 flags.go:64] FLAG: --topology-manager-policy-options=""
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738080    2554 flags.go:64] FLAG: --topology-manager-scope="container"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738083    2554 flags.go:64] FLAG: --v="2"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738088    2554 flags.go:64] FLAG: --version="false"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738092    2554 flags.go:64] FLAG: --vmodule=""
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738096    2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.738099    2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738191    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738195    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738197    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738202    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738205    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738207    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 02:45:11.743924 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738210    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738213    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738215    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738218    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738221    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738223    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738226    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738229    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738231    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738234    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738237    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738240    2554 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738242    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738245    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738247    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738251    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738256    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738258    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738261    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738263    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 02:45:11.744567 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738266    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738268    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738271    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738273    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738276    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738278    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738281    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738283    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738286    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738289    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738292    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738294    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738297    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738300    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738303    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738305    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738308    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738311    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738313    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738316    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 02:45:11.745107 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738318    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738322    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738324    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738327    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738330    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738332    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738335    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738337    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738341    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738343    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738347    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738351    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738354    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738357    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738360    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738363    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738365    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738368    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738371    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738373    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 02:45:11.745624 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738376    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738379    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738381    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738384    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738386    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738389    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738391    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738394    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738396    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738399    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738401    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738404    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738406    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738409    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738411    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738414    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738418    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738420    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738423    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 02:45:11.746128 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.738425    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.739090    2554 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.745538    2554 server.go:530] "Kubelet version" kubeletVersion="v1.33.12"
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.745556    2554 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745605    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745610    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745613    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745616    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745619    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745623    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745626    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745629    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745631    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745634    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745637    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745640    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 02:45:11.746631 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745643    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745645    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745648    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745651    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745653    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745656    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745658    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745661    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745663    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745666    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745669    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745671    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745674    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745679    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745684    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745687    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745690    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745693    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745696    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 02:45:11.747048 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745700    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745703    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745705    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745708    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745711    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745714    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745716    2554 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745719    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745721    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745724    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745726    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745730    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745732    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745735    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745738    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745740    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745742    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745745    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745748    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745751    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 02:45:11.747534 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745753    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745756    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745759    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745761    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745764    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745766    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745769    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745772    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745775    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745777    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745780    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745783    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745786    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745788    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745791    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745793    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745796    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745799    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745801    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745804    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 02:45:11.748012 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745806    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745809    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745811    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745814    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745816    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745821    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745824    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745827    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745830    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745833    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745835    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745838    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745840    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745843    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745846    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 02:45:11.748554 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.745851    2554 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745967    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745972    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745976    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745978    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745981    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745984    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745987    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745989    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745992    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745995    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.745998    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746000    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746003    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746005    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746008    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746011    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746013    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746016    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 02:45:11.748940 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746018    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746022    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746025    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746027    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746030    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746033    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746035    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746038    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746040    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746043    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746046    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746048    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746050    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746053    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746056    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746058    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746060    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746063    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746065    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 02:45:11.749414 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746069    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746071    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746074    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746076    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746079    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746081    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746084    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746086    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746089    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746091    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746094    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746096    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746099    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746101    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746104    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746106    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746108    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746111    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746114    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746116    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 02:45:11.749907 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746119    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746122    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746124    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746126    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746129    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746132    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746134    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746137    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746140    2554 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746142    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746146    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746150    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746153    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746156    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746159    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746162    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746164    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746167    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746170    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746172    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 02:45:11.750380 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746175    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746177    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746180    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746182    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746185    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746187    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746190    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746192    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:11.746195    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.746200    2554 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 02:45:11.750892 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.746928    2554 server.go:962] "Client rotation is on, will bootstrap in background"
Jun 02 02:45:11.751498 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.751483    2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jun 02 02:45:11.752686 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.752674    2554 server.go:1019] "Starting client certificate rotation"
Jun 02 02:45:11.752788 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.752771    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 02 02:45:11.752821 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.752809    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 02 02:45:11.779085 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.779066    2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 02 02:45:11.780783 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.780765    2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 02 02:45:11.797686 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.797664    2554 log.go:25] "Validated CRI v1 runtime API"
Jun 02 02:45:11.803632 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.803616    2554 log.go:25] "Validated CRI v1 image API"
Jun 02 02:45:11.804877 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.804860    2554 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jun 02 02:45:11.809533 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.809513    2554 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e8d23f44-73d3-466a-9663-e005bc125da7:/dev/nvme0n1p4 f0b4c980-d48f-44e6-a792-6ef8825b59dc:/dev/nvme0n1p3]
Jun 02 02:45:11.809592 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.809531    2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jun 02 02:45:11.814159 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.814039    2554 manager.go:217] Machine: {Timestamp:2026-06-02 02:45:11.813552963 +0000 UTC m=+0.430042046 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100369 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27b8303518fe170c2ea9c312e9f926 SystemUUID:ec27b830-3518-fe17-0c2e-a9c312e9f926 BootID:6fe21a72-d671-476f-8f5f-99cff5969a0c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9b:fe:32:a9:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9b:fe:32:a9:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:d1:36:ca:d3:26 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jun 02 02:45:11.814819 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.814802    2554 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 02 02:45:11.815297 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.815286    2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jun 02 02:45:11.815384 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.815372    2554 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jun 02 02:45:11.817357 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.817333    2554 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jun 02 02:45:11.817514 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.817363    2554 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-101.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jun 02 02:45:11.817558 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.817525    2554 topology_manager.go:138] "Creating topology manager with none policy"
Jun 02 02:45:11.817558 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.817534    2554 container_manager_linux.go:306] "Creating device plugin manager"
Jun 02 02:45:11.817558 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.817542    2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 02 02:45:11.818074 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.818065    2554 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 02 02:45:11.819506 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.819495    2554 state_mem.go:36] "Initialized new in-memory state store"
Jun 02 02:45:11.819614 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.819605    2554 server.go:1267] "Using root directory" path="/var/lib/kubelet"
Jun 02 02:45:11.822221 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.822212    2554 kubelet.go:491] "Attempting to sync node with API server"
Jun 02 02:45:11.822254 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.822229    2554 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
Jun 02 02:45:11.822254 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.822241    2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jun 02 02:45:11.822254 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.822249    2554 kubelet.go:397] "Adding apiserver pod source"
Jun 02 02:45:11.822394 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.822258    2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jun 02 02:45:11.823375 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.823362    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 02 02:45:11.823412 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.823383    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 02 02:45:11.826705 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.826691    2554 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1"
Jun 02 02:45:11.828991 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.828979    2554 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jun 02 02:45:11.831484 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831472    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jun 02 02:45:11.831531 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831490    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jun 02 02:45:11.831531 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831497    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jun 02 02:45:11.831531 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831502    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jun 02 02:45:11.831531 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831511    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jun 02 02:45:11.831531 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831517    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jun 02 02:45:11.831531 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831523    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jun 02 02:45:11.831531 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831528    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jun 02 02:45:11.831714 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831535    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jun 02 02:45:11.831714 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831541    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jun 02 02:45:11.831714 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831549    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jun 02 02:45:11.831714 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.831556    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jun 02 02:45:11.832490 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.832479    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jun 02 02:45:11.832540 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.832492    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jun 02 02:45:11.835768 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.835754    2554 watchdog_linux.go:99] "Systemd watchdog is not enabled"
Jun 02 02:45:11.835813 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.835787    2554 server.go:1295] "Started kubelet"
Jun 02 02:45:11.836817 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.836706    2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jun 02 02:45:11.836817 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.836728    2554 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jun 02 02:45:11.836914 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.836823    2554 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jun 02 02:45:11.836958 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.836884    2554 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jun 02 02:45:11.836991 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.836959    2554 server_v1.go:47] "podresources" method="list" useActivePods=true
Jun 02 02:45:11.837543 ip-10-0-129-101 systemd[1]: Started Kubernetes Kubelet.
Jun 02 02:45:11.838116 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.838099    2554 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jun 02 02:45:11.838383 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.838258    2554 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-101.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 02:45:11.840186 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.840168    2554 server.go:317] "Adding debug handlers to kubelet server"
Jun 02 02:45:11.846415 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.846401    2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jun 02 02:45:11.846556 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.846419    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jun 02 02:45:11.846556 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.846512    2554 volume_manager.go:295] "The desired_state_of_world populator starts"
Jun 02 02:45:11.846556 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.846522    2554 volume_manager.go:297] "Starting Kubelet Volume Manager"
Jun 02 02:45:11.846705 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.846590    2554 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
Jun 02 02:45:11.846705 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.846653    2554 reconstruct.go:97] "Volume reconstruction finished"
Jun 02 02:45:11.846705 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.846660    2554 reconciler.go:26] "Reconciler: start to sync state"
Jun 02 02:45:11.847247 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.847230    2554 factory.go:55] Registering systemd factory
Jun 02 02:45:11.847326 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.847253    2554 factory.go:223] Registration of the systemd container factory successfully
Jun 02 02:45:11.847404 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.847385    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:11.847495 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.847479    2554 factory.go:153] Registering CRI-O factory
Jun 02 02:45:11.847546 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.847498    2554 factory.go:223] Registration of the crio container factory successfully
Jun 02 02:45:11.847605 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.847562    2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jun 02 02:45:11.847605 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.847591    2554 factory.go:103] Registering Raw factory
Jun 02 02:45:11.847704 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.847608    2554 manager.go:1196] Started watching for new ooms in manager
Jun 02 02:45:11.847992 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.846956    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-101.ec2.internal.18b5238bfc553227  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-101.ec2.internal,UID:ip-10-0-129-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-101.ec2.internal,},FirstTimestamp:2026-06-02 02:45:11.835767335 +0000 UTC m=+0.452256417,LastTimestamp:2026-06-02 02:45:11.835767335 +0000 UTC m=+0.452256417,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-101.ec2.internal,}"
Jun 02 02:45:11.847992 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.847959    2554 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jun 02 02:45:11.848149 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.848030    2554 manager.go:319] Starting recovery of all containers
Jun 02 02:45:11.858168 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.858053    2554 manager.go:324] Recovery completed
Jun 02 02:45:11.858787 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.858767    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wzs6f"
Jun 02 02:45:11.859568 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.859551    2554 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jun 02 02:45:11.860010 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.859990    2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jun 02 02:45:11.860010 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.860000    2554 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jun 02 02:45:11.862882 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.862869    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 02:45:11.865248 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.865233    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 02:45:11.865311 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.865263    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 02:45:11.865311 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.865274    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID"
Jun 02 02:45:11.865795 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.865782    2554 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jun 02 02:45:11.865795 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.865795    2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jun 02 02:45:11.865915 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.865813    2554 state_mem.go:36] "Initialized new in-memory state store"
Jun 02 02:45:11.866789 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.866775    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wzs6f"
Jun 02 02:45:11.868148 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.868124    2554 policy_none.go:49] "None policy: Start"
Jun 02 02:45:11.868148 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.868145    2554 memory_manager.go:186] "Starting memorymanager" policy="None"
Jun 02 02:45:11.868257 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.868156    2554 state_mem.go:35] "Initializing new in-memory state store"
Jun 02 02:45:11.910993 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.910979    2554 manager.go:341] "Starting Device Plugin manager"
Jun 02 02:45:11.925694 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.911024    2554 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jun 02 02:45:11.925694 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.911034    2554 server.go:85] "Starting device plugin registration server"
Jun 02 02:45:11.925694 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.911282    2554 eviction_manager.go:189] "Eviction manager: starting control loop"
Jun 02 02:45:11.925694 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.911293    2554 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jun 02 02:45:11.925694 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.911409    2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jun 02 02:45:11.925694 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.911497    2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jun 02 02:45:11.925694 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.911507    2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jun 02 02:45:11.925694 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.911984    2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jun 02 02:45:11.925694 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.912026    2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:11.994026 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.993968    2554 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
Jun 02 02:45:11.995111 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.995095    2554 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
Jun 02 02:45:11.995156 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.995123    2554 status_manager.go:230] "Starting to sync pod status with apiserver"
Jun 02 02:45:11.995156 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.995142    2554 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jun 02 02:45:11.995156 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.995150    2554 kubelet.go:2451] "Starting kubelet main sync loop"
Jun 02 02:45:11.995254 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:11.995186    2554 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jun 02 02:45:11.998534 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:11.998512    2554 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 02:45:12.012297 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.012281    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 02:45:12.013106 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.013091    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 02:45:12.013176 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.013123    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 02:45:12.013176 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.013140    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID"
Jun 02 02:45:12.013176 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.013168    2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.022148 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.022133    2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.022198 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.022154    2554 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-101.ec2.internal\": node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.037883 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.037860    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.096079 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.096029    2554 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal"]
Jun 02 02:45:12.096222 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.096123    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 02:45:12.098062 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.098047    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 02:45:12.098131 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.098076    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 02:45:12.098131 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.098086    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID"
Jun 02 02:45:12.099238 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.099226    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 02:45:12.099900 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.099884    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 02:45:12.099953 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.099898    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.099953 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.099918    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 02:45:12.099953 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.099928    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID"
Jun 02 02:45:12.099953 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.099930    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 02:45:12.100566 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.100554    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 02:45:12.100614 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.100578    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 02:45:12.100614 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.100590    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID"
Jun 02 02:45:12.101134 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.101118    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.101230 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.101144    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 02:45:12.101814 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.101794    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 02:45:12.101878 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.101821    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 02:45:12.101878 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.101831    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID"
Jun 02 02:45:12.118998 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.118979    2554 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-101.ec2.internal\" not found" node="ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.120274 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.120257    2554 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-101.ec2.internal\" not found" node="ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.138502 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.138485    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.148057 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.148038    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/775fc32f06cbfbce3c392c36c7bf482e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"775fc32f06cbfbce3c392c36c7bf482e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.148107 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.148065    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/775fc32f06cbfbce3c392c36c7bf482e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"775fc32f06cbfbce3c392c36c7bf482e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.148107 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.148091    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ccd85c0378921da30d2712ae09333302-config\") pod \"kube-apiserver-proxy-ip-10-0-129-101.ec2.internal\" (UID: \"ccd85c0378921da30d2712ae09333302\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.239603 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.239562    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.248962 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.248909    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/775fc32f06cbfbce3c392c36c7bf482e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"775fc32f06cbfbce3c392c36c7bf482e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.248962 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.248935    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/775fc32f06cbfbce3c392c36c7bf482e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"775fc32f06cbfbce3c392c36c7bf482e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.248962 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.248951    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ccd85c0378921da30d2712ae09333302-config\") pod \"kube-apiserver-proxy-ip-10-0-129-101.ec2.internal\" (UID: \"ccd85c0378921da30d2712ae09333302\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.249077 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.249022    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ccd85c0378921da30d2712ae09333302-config\") pod \"kube-apiserver-proxy-ip-10-0-129-101.ec2.internal\" (UID: \"ccd85c0378921da30d2712ae09333302\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.249077 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.249027    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/775fc32f06cbfbce3c392c36c7bf482e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"775fc32f06cbfbce3c392c36c7bf482e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.249077 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.249027    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/775fc32f06cbfbce3c392c36c7bf482e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"775fc32f06cbfbce3c392c36c7bf482e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.340372 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.340333    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.421857 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.421816    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.422935 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.422918    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:12.440864 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.440836    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.541399 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.541333    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.641843 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.641811    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.742293 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.742268    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.752648 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.752635    2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jun 02 02:45:12.752773 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.752759    2554 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 02 02:45:12.843102 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.843029    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:12.847512 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.847498    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jun 02 02:45:12.857678 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.857656    2554 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 02 02:45:12.868784 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.868755    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 02:40:11 +0000 UTC" deadline="2027-11-16 19:02:17.842315312 +0000 UTC"
Jun 02 02:45:12.868784 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.868778    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12784h17m4.973540294s"
Jun 02 02:45:12.882824 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.882800    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zjt9z"
Jun 02 02:45:12.891426 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:12.891408    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zjt9z"
Jun 02 02:45:12.944240 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:12.944205    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:13.013410 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:13.013375    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd85c0378921da30d2712ae09333302.slice/crio-0f8d91e3c33860053a1e3036105fb2dd834535d1a9c880a703c02f29ae224662 WatchSource:0}: Error finding container 0f8d91e3c33860053a1e3036105fb2dd834535d1a9c880a703c02f29ae224662: Status 404 returned error can't find the container with id 0f8d91e3c33860053a1e3036105fb2dd834535d1a9c880a703c02f29ae224662
Jun 02 02:45:13.013708 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:13.013684    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775fc32f06cbfbce3c392c36c7bf482e.slice/crio-73cee30a7dcf2171597d166bad18107e65cbf3a8f2a9ca70719a45523fe99294 WatchSource:0}: Error finding container 73cee30a7dcf2171597d166bad18107e65cbf3a8f2a9ca70719a45523fe99294: Status 404 returned error can't find the container with id 73cee30a7dcf2171597d166bad18107e65cbf3a8f2a9ca70719a45523fe99294
Jun 02 02:45:13.017104 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.017087    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 02:45:13.044596 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:13.044571    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:13.071870 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.071853    2554 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 02:45:13.145291 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:13.145226    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:13.218686 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.218665    2554 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 02:45:13.245886 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:13.245862    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found"
Jun 02 02:45:13.275720 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.275700    2554 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 02:45:13.346561 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.346541    2554 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:13.360063 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.360037    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 02 02:45:13.361112 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.361099    2554 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal"
Jun 02 02:45:13.369491 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.369475    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 02 02:45:13.809217 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.809183    2554 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 02:45:13.823467 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.823436    2554 apiserver.go:52] "Watching apiserver"
Jun 02 02:45:13.833877 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.833712    2554 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jun 02 02:45:13.834156 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.834131    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs","openshift-cluster-node-tuning-operator/tuned-6qxdj","openshift-image-registry/node-ca-4x78s","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal","openshift-multus/multus-vdf4f","openshift-network-diagnostics/network-check-target-bcmlx","openshift-network-operator/iptables-alerter-jqf8v","kube-system/konnectivity-agent-msg6m","kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal","openshift-dns/node-resolver-cn8tf","openshift-multus/multus-additional-cni-plugins-4stzd","openshift-multus/network-metrics-daemon-vnj4x","openshift-ovn-kubernetes/ovnkube-node-ws7xp"]
Jun 02 02:45:13.837260 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.836725    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.837260 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.836754    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.839657 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.839634    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jun 02 02:45:13.839754 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.839722    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jun 02 02:45:13.839916 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.839814    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jun 02 02:45:13.840030 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.839926    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f4q6l\""
Jun 02 02:45:13.840030 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.840022    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wkpx2\""
Jun 02 02:45:13.840362 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.840340    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jun 02 02:45:13.840547 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.840513    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.842865 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.842622    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jun 02 02:45:13.842865 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.842663    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jun 02 02:45:13.842865 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.842674    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.843133 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.843096    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m2wb6\""
Jun 02 02:45:13.843413 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.843391    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jun 02 02:45:13.844067 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.844048    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jun 02 02:45:13.844890 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.844870    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:13.845035 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.845021    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.845035 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:13.845014    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:13.845825 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.845808    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jun 02 02:45:13.845943 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.845874    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c6d95\""
Jun 02 02:45:13.846002 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.845985    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jun 02 02:45:13.846776 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.845818    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jun 02 02:45:13.847588 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.847180    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jun 02 02:45:13.848784 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.847953    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jun 02 02:45:13.848784 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.848059    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jun 02 02:45:13.848784 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.848157    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pvzbp\""
Jun 02 02:45:13.848784 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.848781    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:13.850248 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.850056    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:13.851420 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.851062    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kctnl\""
Jun 02 02:45:13.851420 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.851289    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jun 02 02:45:13.853505 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.852499    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jun 02 02:45:13.853505 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.852763    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pqbcw\""
Jun 02 02:45:13.853505 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.852926    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jun 02 02:45:13.853505 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.853109    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jun 02 02:45:13.854486 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.854470    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jun 02 02:45:13.855489 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.855107    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:13.855489 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:13.855180    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856628    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-run-multus-certs\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856659    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6skl\" (UniqueName: \"kubernetes.io/projected/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-kube-api-access-q6skl\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856706    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd9ef910-5486-460d-af31-0785eefbde99-iptables-alerter-script\") pod \"iptables-alerter-jqf8v\" (UID: \"dd9ef910-5486-460d-af31-0785eefbde99\") " pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856736    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd9ef910-5486-460d-af31-0785eefbde99-host-slash\") pod \"iptables-alerter-jqf8v\" (UID: \"dd9ef910-5486-460d-af31-0785eefbde99\") " pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856767    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt2j8\" (UniqueName: \"kubernetes.io/projected/dd9ef910-5486-460d-af31-0785eefbde99-kube-api-access-wt2j8\") pod \"iptables-alerter-jqf8v\" (UID: \"dd9ef910-5486-460d-af31-0785eefbde99\") " pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856798    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-kubernetes\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856834    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-sysctl-d\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856843    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856860    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/adc9d363-18a3-4d49-a07c-e13c52572722-konnectivity-ca\") pod \"konnectivity-agent-msg6m\" (UID: \"adc9d363-18a3-4d49-a07c-e13c52572722\") " pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856891    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-os-release\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856921    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-run-netns\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.856973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.856953    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-hostroot\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857009    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-sysctl-conf\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857038    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-lib-modules\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857066    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-system-cni-dir\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857091    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-daemon-config\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857167    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-var-lib-kubelet\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857195    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23a084fd-4b0c-4c56-930d-b946a7e1bcbe-host\") pod \"node-ca-4x78s\" (UID: \"23a084fd-4b0c-4c56-930d-b946a7e1bcbe\") " pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857231    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/adc9d363-18a3-4d49-a07c-e13c52572722-agent-certs\") pod \"konnectivity-agent-msg6m\" (UID: \"adc9d363-18a3-4d49-a07c-e13c52572722\") " pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857256    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-cni-binary-copy\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857293    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-var-lib-cni-bin\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857332    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-run\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857362    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-host\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857429    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23a084fd-4b0c-4c56-930d-b946a7e1bcbe-serviceca\") pod \"node-ca-4x78s\" (UID: \"23a084fd-4b0c-4c56-930d-b946a7e1bcbe\") " pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857439    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857507    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b170f48b-6b05-4361-90e3-4f5b81d273ea-tmp-dir\") pod \"node-resolver-cn8tf\" (UID: \"b170f48b-6b05-4361-90e3-4f5b81d273ea\") " pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857550    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-cni-dir\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.857572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857578    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-socket-dir-parent\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.858244 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857733    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:13.858244 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857763    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-modprobe-d\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.858244 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857806    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d426a9ef-8544-48f0-9e47-30f550ef29d1-tmp\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.858244 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857858    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp7nr\" (UniqueName: \"kubernetes.io/projected/d426a9ef-8544-48f0-9e47-30f550ef29d1-kube-api-access-lp7nr\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.858244 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857888    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-conf-dir\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.858244 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.857934    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-tuned\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.859069 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.858996    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8vfn\" (UniqueName: \"kubernetes.io/projected/23a084fd-4b0c-4c56-930d-b946a7e1bcbe-kube-api-access-d8vfn\") pod \"node-ca-4x78s\" (UID: \"23a084fd-4b0c-4c56-930d-b946a7e1bcbe\") " pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.859069 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859047    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b170f48b-6b05-4361-90e3-4f5b81d273ea-hosts-file\") pod \"node-resolver-cn8tf\" (UID: \"b170f48b-6b05-4361-90e3-4f5b81d273ea\") " pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.859215 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859140    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnj6l\" (UniqueName: \"kubernetes.io/projected/b170f48b-6b05-4361-90e3-4f5b81d273ea-kube-api-access-qnj6l\") pod \"node-resolver-cn8tf\" (UID: \"b170f48b-6b05-4361-90e3-4f5b81d273ea\") " pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.859215 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859180    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-run-k8s-cni-cncf-io\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.859324 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859222    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-var-lib-cni-multus\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.859324 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859257    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-var-lib-kubelet\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.859324 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859292    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-sysconfig\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.859324 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859319    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-systemd\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.859566 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859351    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-sys\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.859566 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859396    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-cnibin\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.859566 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.859433    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-etc-kubernetes\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.861934 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.861906    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jun 02 02:45:13.862645 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.862534    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jun 02 02:45:13.862645 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.862599    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jun 02 02:45:13.862794 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.862746    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jun 02 02:45:13.864721 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.864673    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jun 02 02:45:13.864812 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.864721    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cbw5t\""
Jun 02 02:45:13.864867 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.864806    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jun 02 02:45:13.865648 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.865624    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jun 02 02:45:13.865737 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.864925    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jun 02 02:45:13.865737 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.865701    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m86qz\""
Jun 02 02:45:13.893740 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.893715    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 02:40:12 +0000 UTC" deadline="2027-12-30 00:47:37.408948544 +0000 UTC"
Jun 02 02:45:13.893740 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.893739    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13822h2m23.515212297s"
Jun 02 02:45:13.947478 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.947440    2554 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
Jun 02 02:45:13.960609 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960579    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wt2j8\" (UniqueName: \"kubernetes.io/projected/dd9ef910-5486-460d-af31-0785eefbde99-kube-api-access-wt2j8\") pod \"iptables-alerter-jqf8v\" (UID: \"dd9ef910-5486-460d-af31-0785eefbde99\") " pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.960717 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960628    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-sysctl-d\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.960779 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960700    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbprb\" (UniqueName: \"kubernetes.io/projected/fcfae5f4-8103-45fa-beee-bdb243b5ee53-kube-api-access-gbprb\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:13.960779 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960751    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-var-lib-openvswitch\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.960874 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960783    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/59e50745-ea12-4b4d-9707-0776345bfe9c-ovnkube-config\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.960874 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960794    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-sysctl-d\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.960874 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960805    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:13.960874 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960834    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/007548f0-391f-4fc2-8977-615a8a04f403-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:13.961064 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960883    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-os-release\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961064 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960925    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-run-netns\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961064 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960958    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-sys-fs\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:13.961064 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.960963    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-os-release\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961064 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961013    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-run-netns\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961064 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961013    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-system-cni-dir\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:13.961327 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961065    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-system-cni-dir\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961327 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961094    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23a084fd-4b0c-4c56-930d-b946a7e1bcbe-host\") pod \"node-ca-4x78s\" (UID: \"23a084fd-4b0c-4c56-930d-b946a7e1bcbe\") " pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.961327 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961146    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/adc9d363-18a3-4d49-a07c-e13c52572722-agent-certs\") pod \"konnectivity-agent-msg6m\" (UID: \"adc9d363-18a3-4d49-a07c-e13c52572722\") " pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:13.961327 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961186    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-system-cni-dir\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961327 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961189    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-run-netns\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.961327 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961229    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6mdc\" (UniqueName: \"kubernetes.io/projected/007548f0-391f-4fc2-8977-615a8a04f403-kube-api-access-d6mdc\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:13.961327 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961239    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23a084fd-4b0c-4c56-930d-b946a7e1bcbe-host\") pod \"node-ca-4x78s\" (UID: \"23a084fd-4b0c-4c56-930d-b946a7e1bcbe\") " pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.961327 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961263    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-var-lib-cni-bin\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961333    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-host\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961364    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-run-systemd\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961398    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-os-release\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961426    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-cni-dir\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961468    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-socket-dir-parent\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961506    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-host\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961495    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961552    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d426a9ef-8544-48f0-9e47-30f550ef29d1-tmp\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961603    2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961689    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lp7nr\" (UniqueName: \"kubernetes.io/projected/d426a9ef-8544-48f0-9e47-30f550ef29d1-kube-api-access-lp7nr\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961723    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-kubelet\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961714    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-cni-dir\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.961755 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961752    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-socket-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961792    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-log-socket\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961805    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-var-lib-cni-bin\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961808    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-socket-dir-parent\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961816    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-cni-netd\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961946    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b170f48b-6b05-4361-90e3-4f5b81d273ea-hosts-file\") pod \"node-resolver-cn8tf\" (UID: \"b170f48b-6b05-4361-90e3-4f5b81d273ea\") " pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.961993    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qnj6l\" (UniqueName: \"kubernetes.io/projected/b170f48b-6b05-4361-90e3-4f5b81d273ea-kube-api-access-qnj6l\") pod \"node-resolver-cn8tf\" (UID: \"b170f48b-6b05-4361-90e3-4f5b81d273ea\") " pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962019    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b170f48b-6b05-4361-90e3-4f5b81d273ea-hosts-file\") pod \"node-resolver-cn8tf\" (UID: \"b170f48b-6b05-4361-90e3-4f5b81d273ea\") " pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962059    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-var-lib-cni-multus\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962094    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-sysconfig\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962142    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-var-lib-cni-multus\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962185    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-sysconfig\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962217    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/59e50745-ea12-4b4d-9707-0776345bfe9c-env-overrides\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962246    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-registration-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962271    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-cnibin\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962292    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-run-ovn-kubernetes\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.962318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962315    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962338    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ncgx\" (UniqueName: \"kubernetes.io/projected/1120f884-f9f1-41a3-886c-5fe2001bf3f3-kube-api-access-9ncgx\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962371    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962393    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/007548f0-391f-4fc2-8977-615a8a04f403-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962411    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-cnibin\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962421    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-run-multus-certs\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962447    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q6skl\" (UniqueName: \"kubernetes.io/projected/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-kube-api-access-q6skl\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962507    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-run-multus-certs\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962510    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd9ef910-5486-460d-af31-0785eefbde99-iptables-alerter-script\") pod \"iptables-alerter-jqf8v\" (UID: \"dd9ef910-5486-460d-af31-0785eefbde99\") " pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962555    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd9ef910-5486-460d-af31-0785eefbde99-host-slash\") pod \"iptables-alerter-jqf8v\" (UID: \"dd9ef910-5486-460d-af31-0785eefbde99\") " pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962668    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd9ef910-5486-460d-af31-0785eefbde99-host-slash\") pod \"iptables-alerter-jqf8v\" (UID: \"dd9ef910-5486-460d-af31-0785eefbde99\") " pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962704    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-kubernetes\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962752    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-kubernetes\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962775    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/adc9d363-18a3-4d49-a07c-e13c52572722-konnectivity-ca\") pod \"konnectivity-agent-msg6m\" (UID: \"adc9d363-18a3-4d49-a07c-e13c52572722\") " pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962808    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-systemd-units\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962833    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xrcp\" (UniqueName: \"kubernetes.io/projected/59e50745-ea12-4b4d-9707-0776345bfe9c-kube-api-access-4xrcp\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962858    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-hostroot\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.963136 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962929    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-sysctl-conf\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962960    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-lib-modules\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.962990    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-cni-bin\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963021    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-daemon-config\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963056    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd9ef910-5486-460d-af31-0785eefbde99-iptables-alerter-script\") pod \"iptables-alerter-jqf8v\" (UID: \"dd9ef910-5486-460d-af31-0785eefbde99\") " pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963060    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-var-lib-kubelet\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963103    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-run-openvswitch\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963150    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/007548f0-391f-4fc2-8977-615a8a04f403-cni-binary-copy\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963178    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-cni-binary-copy\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963220    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-sysctl-conf\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963228    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-run\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963261    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23a084fd-4b0c-4c56-930d-b946a7e1bcbe-serviceca\") pod \"node-ca-4x78s\" (UID: \"23a084fd-4b0c-4c56-930d-b946a7e1bcbe\") " pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963285    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-run\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963329    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-hostroot\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963416    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-lib-modules\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963287    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-etc-openvswitch\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963572    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-var-lib-kubelet\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.963924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963750    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23a084fd-4b0c-4c56-930d-b946a7e1bcbe-serviceca\") pod \"node-ca-4x78s\" (UID: \"23a084fd-4b0c-4c56-930d-b946a7e1bcbe\") " pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963823    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/59e50745-ea12-4b4d-9707-0776345bfe9c-ovnkube-script-lib\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963853    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/adc9d363-18a3-4d49-a07c-e13c52572722-konnectivity-ca\") pod \"konnectivity-agent-msg6m\" (UID: \"adc9d363-18a3-4d49-a07c-e13c52572722\") " pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963865    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b170f48b-6b05-4361-90e3-4f5b81d273ea-tmp-dir\") pod \"node-resolver-cn8tf\" (UID: \"b170f48b-6b05-4361-90e3-4f5b81d273ea\") " pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963889    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-modprobe-d\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963913    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-run-ovn\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.963994    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-conf-dir\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964081    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-cni-binary-copy\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964100    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-daemon-config\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964064    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-tuned\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964164    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-multus-conf-dir\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964169    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b170f48b-6b05-4361-90e3-4f5b81d273ea-tmp-dir\") pod \"node-resolver-cn8tf\" (UID: \"b170f48b-6b05-4361-90e3-4f5b81d273ea\") " pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964187    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-modprobe-d\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964216    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d8vfn\" (UniqueName: \"kubernetes.io/projected/23a084fd-4b0c-4c56-930d-b946a7e1bcbe-kube-api-access-d8vfn\") pod \"node-ca-4x78s\" (UID: \"23a084fd-4b0c-4c56-930d-b946a7e1bcbe\") " pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964256    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-slash\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964329    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-device-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964365    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-run-k8s-cni-cncf-io\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964414    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-var-lib-kubelet\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.964543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964439    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-systemd\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964490    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-run-k8s-cni-cncf-io\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964491    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-host-var-lib-kubelet\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964511    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-sys\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964513    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-systemd\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964569    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d426a9ef-8544-48f0-9e47-30f550ef29d1-sys\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964605    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-node-log\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964631    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-cnibin\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964659    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-etc-kubernetes\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964684    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964707    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/59e50745-ea12-4b4d-9707-0776345bfe9c-ovn-node-metrics-cert\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964743    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-etc-kubernetes\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:13.965322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.965232    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d426a9ef-8544-48f0-9e47-30f550ef29d1-tmp\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.965901 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.965414    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/adc9d363-18a3-4d49-a07c-e13c52572722-agent-certs\") pod \"konnectivity-agent-msg6m\" (UID: \"adc9d363-18a3-4d49-a07c-e13c52572722\") " pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:13.966975 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.964733    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:13.967112 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.967049    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d426a9ef-8544-48f0-9e47-30f550ef29d1-etc-tuned\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.973366 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:13.973341    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 02:45:13.973492 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:13.973386    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 02:45:13.973492 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:13.973404    2554 projected.go:194] Error preparing data for projected volume kube-api-access-bxpdl for pod openshift-network-diagnostics/network-check-target-bcmlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:13.973605 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:13.973541    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl podName:861af5a6-480a-4494-89c0-0d4416a9fb9f nodeName:}" failed. No retries permitted until 2026-06-02 02:45:14.473482677 +0000 UTC m=+3.089971761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bxpdl" (UniqueName: "kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl") pod "network-check-target-bcmlx" (UID: "861af5a6-480a-4494-89c0-0d4416a9fb9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:13.975164 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.975138    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt2j8\" (UniqueName: \"kubernetes.io/projected/dd9ef910-5486-460d-af31-0785eefbde99-kube-api-access-wt2j8\") pod \"iptables-alerter-jqf8v\" (UID: \"dd9ef910-5486-460d-af31-0785eefbde99\") " pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:13.976664 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.976630    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp7nr\" (UniqueName: \"kubernetes.io/projected/d426a9ef-8544-48f0-9e47-30f550ef29d1-kube-api-access-lp7nr\") pod \"tuned-6qxdj\" (UID: \"d426a9ef-8544-48f0-9e47-30f550ef29d1\") " pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:13.978987 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.978949    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8vfn\" (UniqueName: \"kubernetes.io/projected/23a084fd-4b0c-4c56-930d-b946a7e1bcbe-kube-api-access-d8vfn\") pod \"node-ca-4x78s\" (UID: \"23a084fd-4b0c-4c56-930d-b946a7e1bcbe\") " pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:13.978987 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.978960    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnj6l\" (UniqueName: \"kubernetes.io/projected/b170f48b-6b05-4361-90e3-4f5b81d273ea-kube-api-access-qnj6l\") pod \"node-resolver-cn8tf\" (UID: \"b170f48b-6b05-4361-90e3-4f5b81d273ea\") " pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:13.981016 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:13.980995    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6skl\" (UniqueName: \"kubernetes.io/projected/0b560b06-bdcc-4b2b-8cb4-ce91d50a8341-kube-api-access-q6skl\") pod \"multus-vdf4f\" (UID: \"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341\") " pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:14.001158 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.001102    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" event={"ID":"ccd85c0378921da30d2712ae09333302","Type":"ContainerStarted","Data":"0f8d91e3c33860053a1e3036105fb2dd834535d1a9c880a703c02f29ae224662"}
Jun 02 02:45:14.002707 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.002684    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" event={"ID":"775fc32f06cbfbce3c392c36c7bf482e","Type":"ContainerStarted","Data":"73cee30a7dcf2171597d166bad18107e65cbf3a8f2a9ca70719a45523fe99294"}
Jun 02 02:45:14.067442 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067367    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-run-ovn\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.067442 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067408    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-slash\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.067442 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067424    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-device-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.067442 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067439    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-node-log\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067470    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-cnibin\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067509    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067515    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-node-log\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067532    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/59e50745-ea12-4b4d-9707-0776345bfe9c-ovn-node-metrics-cert\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067514    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-run-ovn\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067543    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-slash\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067543    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-device-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067555    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067609    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-cnibin\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067620    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gbprb\" (UniqueName: \"kubernetes.io/projected/fcfae5f4-8103-45fa-beee-bdb243b5ee53-kube-api-access-gbprb\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067651    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:14.067663    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067667    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-var-lib-openvswitch\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067698    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/59e50745-ea12-4b4d-9707-0776345bfe9c-ovnkube-config\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067725    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.067767 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067754    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/007548f0-391f-4fc2-8977-615a8a04f403-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067799    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-sys-fs\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067824    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-system-cni-dir\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067854    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-run-netns\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067763    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-var-lib-openvswitch\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067877    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d6mdc\" (UniqueName: \"kubernetes.io/projected/007548f0-391f-4fc2-8977-615a8a04f403-kube-api-access-d6mdc\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067920    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-run-systemd\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067932    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067947    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-sys-fs\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.067948    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-os-release\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068005    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-run-systemd\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068034    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-os-release\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068071    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-system-cni-dir\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068103    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-run-netns\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068187    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-kubelet\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068213    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-socket-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068240    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-log-socket\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.068594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068263    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-cni-netd\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068292    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/59e50745-ea12-4b4d-9707-0776345bfe9c-env-overrides\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068316    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-registration-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068318    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-socket-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068351    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/59e50745-ea12-4b4d-9707-0776345bfe9c-ovnkube-config\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068368    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1120f884-f9f1-41a3-886c-5fe2001bf3f3-registration-dir\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068367    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-log-socket\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068318    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-kubelet\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068391    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-run-ovn-kubernetes\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068414    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-cni-netd\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068425    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068477    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9ncgx\" (UniqueName: \"kubernetes.io/projected/1120f884-f9f1-41a3-886c-5fe2001bf3f3-kube-api-access-9ncgx\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068478    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-run-ovn-kubernetes\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068509    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:14.068531    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs podName:fcfae5f4-8103-45fa-beee-bdb243b5ee53 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:14.568488243 +0000 UTC m=+3.184977337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs") pod "network-metrics-daemon-vnj4x" (UID: "fcfae5f4-8103-45fa-beee-bdb243b5ee53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068561    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.069353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068579    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/007548f0-391f-4fc2-8977-615a8a04f403-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068588    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/007548f0-391f-4fc2-8977-615a8a04f403-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068647    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-systemd-units\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068676    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4xrcp\" (UniqueName: \"kubernetes.io/projected/59e50745-ea12-4b4d-9707-0776345bfe9c-kube-api-access-4xrcp\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068686    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-systemd-units\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068704    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-cni-bin\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068709    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/007548f0-391f-4fc2-8977-615a8a04f403-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068756    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-host-cni-bin\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068776    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/59e50745-ea12-4b4d-9707-0776345bfe9c-env-overrides\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068876    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-run-openvswitch\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068909    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/007548f0-391f-4fc2-8977-615a8a04f403-cni-binary-copy\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068917    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-run-openvswitch\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068942    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-etc-openvswitch\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.068969    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/59e50745-ea12-4b4d-9707-0776345bfe9c-ovnkube-script-lib\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.069017    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/59e50745-ea12-4b4d-9707-0776345bfe9c-etc-openvswitch\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.069073    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/007548f0-391f-4fc2-8977-615a8a04f403-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.070108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.069337    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/007548f0-391f-4fc2-8977-615a8a04f403-cni-binary-copy\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.070955 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.070334    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/59e50745-ea12-4b4d-9707-0776345bfe9c-ovnkube-script-lib\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.071185 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.071164    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/59e50745-ea12-4b4d-9707-0776345bfe9c-ovn-node-metrics-cert\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.079842 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.079742    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbprb\" (UniqueName: \"kubernetes.io/projected/fcfae5f4-8103-45fa-beee-bdb243b5ee53-kube-api-access-gbprb\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:14.079842 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.079797    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ncgx\" (UniqueName: \"kubernetes.io/projected/1120f884-f9f1-41a3-886c-5fe2001bf3f3-kube-api-access-9ncgx\") pod \"aws-ebs-csi-driver-node-crlcs\" (UID: \"1120f884-f9f1-41a3-886c-5fe2001bf3f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.080000 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.079905    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6mdc\" (UniqueName: \"kubernetes.io/projected/007548f0-391f-4fc2-8977-615a8a04f403-kube-api-access-d6mdc\") pod \"multus-additional-cni-plugins-4stzd\" (UID: \"007548f0-391f-4fc2-8977-615a8a04f403\") " pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.080695 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.080667    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xrcp\" (UniqueName: \"kubernetes.io/projected/59e50745-ea12-4b4d-9707-0776345bfe9c-kube-api-access-4xrcp\") pod \"ovnkube-node-ws7xp\" (UID: \"59e50745-ea12-4b4d-9707-0776345bfe9c\") " pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.154736 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.154705    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cn8tf"
Jun 02 02:45:14.163511 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.163490    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6qxdj"
Jun 02 02:45:14.175040 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.175019    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vdf4f"
Jun 02 02:45:14.179542 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.179522    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4x78s"
Jun 02 02:45:14.188145 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.188126    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jqf8v"
Jun 02 02:45:14.194830 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.194797    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:14.206353 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.206334    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs"
Jun 02 02:45:14.212955 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.212935    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:14.217498 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.217480    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4stzd"
Jun 02 02:45:14.573261 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.573229    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:14.573433 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.573282    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:14.573433 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:14.573419    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:14.573555 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:14.573441    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 02:45:14.573555 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:14.573472    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 02:45:14.573555 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:14.573485    2554 projected.go:194] Error preparing data for projected volume kube-api-access-bxpdl for pod openshift-network-diagnostics/network-check-target-bcmlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:14.573555 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:14.573519    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs podName:fcfae5f4-8103-45fa-beee-bdb243b5ee53 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:15.573497995 +0000 UTC m=+4.189987067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs") pod "network-metrics-daemon-vnj4x" (UID: "fcfae5f4-8103-45fa-beee-bdb243b5ee53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:14.573555 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:14.573542    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl podName:861af5a6-480a-4494-89c0-0d4416a9fb9f nodeName:}" failed. No retries permitted until 2026-06-02 02:45:15.573530849 +0000 UTC m=+4.190019923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bxpdl" (UniqueName: "kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl") pod "network-check-target-bcmlx" (UID: "861af5a6-480a-4494-89c0-0d4416a9fb9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:14.894231 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.894199    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 02:40:12 +0000 UTC" deadline="2028-01-17 06:31:11.2955034 +0000 UTC"
Jun 02 02:45:14.894231 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:14.894227    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14259h45m56.401278358s"
Jun 02 02:45:14.904203 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:14.904173    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd426a9ef_8544_48f0_9e47_30f550ef29d1.slice/crio-d76860e9a7fd448800a2f5ccd4bdb2c1bbde7c0a54f34d7ec2f28bed774dc0b7 WatchSource:0}: Error finding container d76860e9a7fd448800a2f5ccd4bdb2c1bbde7c0a54f34d7ec2f28bed774dc0b7: Status 404 returned error can't find the container with id d76860e9a7fd448800a2f5ccd4bdb2c1bbde7c0a54f34d7ec2f28bed774dc0b7
Jun 02 02:45:14.910002 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:14.909945    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007548f0_391f_4fc2_8977_615a8a04f403.slice/crio-231b8205efcfe111864fdd80d303d833eb4c8296abc1171a082a496a246c938d WatchSource:0}: Error finding container 231b8205efcfe111864fdd80d303d833eb4c8296abc1171a082a496a246c938d: Status 404 returned error can't find the container with id 231b8205efcfe111864fdd80d303d833eb4c8296abc1171a082a496a246c938d
Jun 02 02:45:14.912878 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:14.912833    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59e50745_ea12_4b4d_9707_0776345bfe9c.slice/crio-843e88a94d4aa753f856de5fa6dffcd3937832eba8d9c8fea63ec6c1e3689b97 WatchSource:0}: Error finding container 843e88a94d4aa753f856de5fa6dffcd3937832eba8d9c8fea63ec6c1e3689b97: Status 404 returned error can't find the container with id 843e88a94d4aa753f856de5fa6dffcd3937832eba8d9c8fea63ec6c1e3689b97
Jun 02 02:45:14.913809 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:14.913642    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc9d363_18a3_4d49_a07c_e13c52572722.slice/crio-dfb8a81ef341170e9cc94d6a6b6d3f89c7872b67ce38d252dafceb4ad5778d86 WatchSource:0}: Error finding container dfb8a81ef341170e9cc94d6a6b6d3f89c7872b67ce38d252dafceb4ad5778d86: Status 404 returned error can't find the container with id dfb8a81ef341170e9cc94d6a6b6d3f89c7872b67ce38d252dafceb4ad5778d86
Jun 02 02:45:14.914718 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:14.914607    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9ef910_5486_460d_af31_0785eefbde99.slice/crio-f58887761a7abd83e202a9be1da1d7d74a09a8fa5ce759cf0b4261334224c10e WatchSource:0}: Error finding container f58887761a7abd83e202a9be1da1d7d74a09a8fa5ce759cf0b4261334224c10e: Status 404 returned error can't find the container with id f58887761a7abd83e202a9be1da1d7d74a09a8fa5ce759cf0b4261334224c10e
Jun 02 02:45:14.916483 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:14.916411    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1120f884_f9f1_41a3_886c_5fe2001bf3f3.slice/crio-13edebb268d41f42c2cb76f034a2ade63dadeb66f24c533734068a5da2cb449a WatchSource:0}: Error finding container 13edebb268d41f42c2cb76f034a2ade63dadeb66f24c533734068a5da2cb449a: Status 404 returned error can't find the container with id 13edebb268d41f42c2cb76f034a2ade63dadeb66f24c533734068a5da2cb449a
Jun 02 02:45:14.917187 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:14.917163    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b560b06_bdcc_4b2b_8cb4_ce91d50a8341.slice/crio-e2a4834553dc1108004aa327d6d192e1273cd5038e74dbe8d530d194d7bad999 WatchSource:0}: Error finding container e2a4834553dc1108004aa327d6d192e1273cd5038e74dbe8d530d194d7bad999: Status 404 returned error can't find the container with id e2a4834553dc1108004aa327d6d192e1273cd5038e74dbe8d530d194d7bad999
Jun 02 02:45:14.919405 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:14.919371    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a084fd_4b0c_4c56_930d_b946a7e1bcbe.slice/crio-d4031e946d7fa16a6f78a9cbc1ec4dbe280fa1893f3da195b35ec2655860d11a WatchSource:0}: Error finding container d4031e946d7fa16a6f78a9cbc1ec4dbe280fa1893f3da195b35ec2655860d11a: Status 404 returned error can't find the container with id d4031e946d7fa16a6f78a9cbc1ec4dbe280fa1893f3da195b35ec2655860d11a
Jun 02 02:45:14.920202 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:14.920144    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb170f48b_6b05_4361_90e3_4f5b81d273ea.slice/crio-ae95ea08fdc76f12ac6e4a761547724ccc5bfd4047cb8f1c50ea60ad548e8f24 WatchSource:0}: Error finding container ae95ea08fdc76f12ac6e4a761547724ccc5bfd4047cb8f1c50ea60ad548e8f24: Status 404 returned error can't find the container with id ae95ea08fdc76f12ac6e4a761547724ccc5bfd4047cb8f1c50ea60ad548e8f24
Jun 02 02:45:15.006291 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.006138    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6qxdj" event={"ID":"d426a9ef-8544-48f0-9e47-30f550ef29d1","Type":"ContainerStarted","Data":"d76860e9a7fd448800a2f5ccd4bdb2c1bbde7c0a54f34d7ec2f28bed774dc0b7"}
Jun 02 02:45:15.009124 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.009091    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" event={"ID":"ccd85c0378921da30d2712ae09333302","Type":"ContainerStarted","Data":"582a1f6d9103a46e123242b2620c3dcafd6a9a0fb09b2f40e66fe6477190524d"}
Jun 02 02:45:15.010157 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.010129    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4x78s" event={"ID":"23a084fd-4b0c-4c56-930d-b946a7e1bcbe","Type":"ContainerStarted","Data":"d4031e946d7fa16a6f78a9cbc1ec4dbe280fa1893f3da195b35ec2655860d11a"}
Jun 02 02:45:15.012843 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.012814    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vdf4f" event={"ID":"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341","Type":"ContainerStarted","Data":"e2a4834553dc1108004aa327d6d192e1273cd5038e74dbe8d530d194d7bad999"}
Jun 02 02:45:15.018049 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.018029    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs" event={"ID":"1120f884-f9f1-41a3-886c-5fe2001bf3f3","Type":"ContainerStarted","Data":"13edebb268d41f42c2cb76f034a2ade63dadeb66f24c533734068a5da2cb449a"}
Jun 02 02:45:15.020950 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.020928    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jqf8v" event={"ID":"dd9ef910-5486-460d-af31-0785eefbde99","Type":"ContainerStarted","Data":"f58887761a7abd83e202a9be1da1d7d74a09a8fa5ce759cf0b4261334224c10e"}
Jun 02 02:45:15.021819 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.021797    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-msg6m" event={"ID":"adc9d363-18a3-4d49-a07c-e13c52572722","Type":"ContainerStarted","Data":"dfb8a81ef341170e9cc94d6a6b6d3f89c7872b67ce38d252dafceb4ad5778d86"}
Jun 02 02:45:15.022084 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.022045    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" podStartSLOduration=2.022034328 podStartE2EDuration="2.022034328s" podCreationTimestamp="2026-06-02 02:45:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:45:15.021758026 +0000 UTC m=+3.638247110" watchObservedRunningTime="2026-06-02 02:45:15.022034328 +0000 UTC m=+3.638523418"
Jun 02 02:45:15.025990 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.025974    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerStarted","Data":"843e88a94d4aa753f856de5fa6dffcd3937832eba8d9c8fea63ec6c1e3689b97"}
Jun 02 02:45:15.026962 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.026942    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cn8tf" event={"ID":"b170f48b-6b05-4361-90e3-4f5b81d273ea","Type":"ContainerStarted","Data":"ae95ea08fdc76f12ac6e4a761547724ccc5bfd4047cb8f1c50ea60ad548e8f24"}
Jun 02 02:45:15.030442 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.030418    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4stzd" event={"ID":"007548f0-391f-4fc2-8977-615a8a04f403","Type":"ContainerStarted","Data":"231b8205efcfe111864fdd80d303d833eb4c8296abc1171a082a496a246c938d"}
Jun 02 02:45:15.580988 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.580900    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:15.581148 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.580996    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:15.581148 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:15.581132    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:15.581260 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:15.581199    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs podName:fcfae5f4-8103-45fa-beee-bdb243b5ee53 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:17.581177161 +0000 UTC m=+6.197666234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs") pod "network-metrics-daemon-vnj4x" (UID: "fcfae5f4-8103-45fa-beee-bdb243b5ee53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:15.581845 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:15.581668    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 02:45:15.581845 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:15.581694    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 02:45:15.581845 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:15.581707    2554 projected.go:194] Error preparing data for projected volume kube-api-access-bxpdl for pod openshift-network-diagnostics/network-check-target-bcmlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:15.581845 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:15.581804    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl podName:861af5a6-480a-4494-89c0-0d4416a9fb9f nodeName:}" failed. No retries permitted until 2026-06-02 02:45:17.581742071 +0000 UTC m=+6.198231149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bxpdl" (UniqueName: "kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl") pod "network-check-target-bcmlx" (UID: "861af5a6-480a-4494-89c0-0d4416a9fb9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:15.997917 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.997888    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:15.998375 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:15.998024    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:15.998469 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:15.998431    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:15.998603 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:15.998551    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:16.049986 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:16.048795    2554 generic.go:358] "Generic (PLEG): container finished" podID="775fc32f06cbfbce3c392c36c7bf482e" containerID="80ceeb71a096affc00b3dfb76e2769b7d3298f4f2fbb6e55d0bcc1dc2bf93d4d" exitCode=0
Jun 02 02:45:16.049986 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:16.049715    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" event={"ID":"775fc32f06cbfbce3c392c36c7bf482e","Type":"ContainerDied","Data":"80ceeb71a096affc00b3dfb76e2769b7d3298f4f2fbb6e55d0bcc1dc2bf93d4d"}
Jun 02 02:45:17.079224 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:17.078398    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" event={"ID":"775fc32f06cbfbce3c392c36c7bf482e","Type":"ContainerStarted","Data":"4f287d8047d1e41c1589068795bd18cef5bbbcceab586caa0c33a96b433d3fb3"}
Jun 02 02:45:17.598883 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:17.598846    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:17.599055 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:17.598946    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:17.599128 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:17.599091    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:17.599187 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:17.599156    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs podName:fcfae5f4-8103-45fa-beee-bdb243b5ee53 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:21.599138186 +0000 UTC m=+10.215627268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs") pod "network-metrics-daemon-vnj4x" (UID: "fcfae5f4-8103-45fa-beee-bdb243b5ee53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:17.599690 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:17.599669    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 02:45:17.599783 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:17.599699    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 02:45:17.599783 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:17.599713    2554 projected.go:194] Error preparing data for projected volume kube-api-access-bxpdl for pod openshift-network-diagnostics/network-check-target-bcmlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:17.599783 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:17.599763    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl podName:861af5a6-480a-4494-89c0-0d4416a9fb9f nodeName:}" failed. No retries permitted until 2026-06-02 02:45:21.599746832 +0000 UTC m=+10.216235907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bxpdl" (UniqueName: "kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl") pod "network-check-target-bcmlx" (UID: "861af5a6-480a-4494-89c0-0d4416a9fb9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:17.996275 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:17.996185    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:17.996438 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:17.996324    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:17.996707 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:17.996189    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:17.996833 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:17.996810    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:19.996528 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:19.996430    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:19.996528 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:19.996484    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:19.997053 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:19.996607    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:19.997053 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:19.996702    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:20.684088 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.683030    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" podStartSLOduration=7.683011744 podStartE2EDuration="7.683011744s" podCreationTimestamp="2026-06-02 02:45:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:45:17.092645031 +0000 UTC m=+5.709134122" watchObservedRunningTime="2026-06-02 02:45:20.683011744 +0000 UTC m=+9.299500836"
Jun 02 02:45:20.684088 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.683353    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-fln9v"]
Jun 02 02:45:20.691085 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.690741    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:20.691085 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:20.690828    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:20.824511 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.824469    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-kubelet-config\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:20.824685 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.824547    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-dbus\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:20.824685 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.824610    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:20.925380 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.925341    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:20.925568 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.925408    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-kubelet-config\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:20.925568 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.925480    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-dbus\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:20.925568 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:20.925538    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:20.925685 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.925575    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-kubelet-config\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:20.925685 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:20.925612    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret podName:7093c19d-4a2c-4d50-aada-46c00f5c5dd7 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:21.42559074 +0000 UTC m=+10.042079812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret") pod "global-pull-secret-syncer-fln9v" (UID: "7093c19d-4a2c-4d50-aada-46c00f5c5dd7") : object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:20.925685 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:20.925656    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-dbus\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:21.429152 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:21.429093    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:21.429626 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.429225    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:21.429626 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.429399    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret podName:7093c19d-4a2c-4d50-aada-46c00f5c5dd7 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:22.429276578 +0000 UTC m=+11.045765698 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret") pod "global-pull-secret-syncer-fln9v" (UID: "7093c19d-4a2c-4d50-aada-46c00f5c5dd7") : object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:21.630430 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:21.630390    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:21.630629 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:21.630486    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:21.630693 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.630638    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 02:45:21.630693 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.630656    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 02:45:21.630693 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.630669    2554 projected.go:194] Error preparing data for projected volume kube-api-access-bxpdl for pod openshift-network-diagnostics/network-check-target-bcmlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:21.630837 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.630728    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl podName:861af5a6-480a-4494-89c0-0d4416a9fb9f nodeName:}" failed. No retries permitted until 2026-06-02 02:45:29.630709436 +0000 UTC m=+18.247198512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bxpdl" (UniqueName: "kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl") pod "network-check-target-bcmlx" (UID: "861af5a6-480a-4494-89c0-0d4416a9fb9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:21.631061 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.631038    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:21.631142 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.631105    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs podName:fcfae5f4-8103-45fa-beee-bdb243b5ee53 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:29.631086431 +0000 UTC m=+18.247575506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs") pod "network-metrics-daemon-vnj4x" (UID: "fcfae5f4-8103-45fa-beee-bdb243b5ee53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:21.995669 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:21.995633    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:21.996677 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:21.996591    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:21.996804 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.996692    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:21.996804 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:21.996784    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:21.996921 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.996888    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:21.996988 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:21.996963    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:22.436872 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:22.436828    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:22.437328 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:22.436984    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:22.437328 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:22.437061    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret podName:7093c19d-4a2c-4d50-aada-46c00f5c5dd7 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:24.437040084 +0000 UTC m=+13.053529158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret") pod "global-pull-secret-syncer-fln9v" (UID: "7093c19d-4a2c-4d50-aada-46c00f5c5dd7") : object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:23.996094 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:23.995606    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:23.996094 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:23.995607    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:23.996094 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:23.995754    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:23.996094 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:23.995822    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:23.996094 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:23.995831    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:23.996094 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:23.995919    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:24.453194 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:24.453105    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:24.453347 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:24.453273    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:24.453347 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:24.453347    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret podName:7093c19d-4a2c-4d50-aada-46c00f5c5dd7 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:28.453332643 +0000 UTC m=+17.069821712 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret") pod "global-pull-secret-syncer-fln9v" (UID: "7093c19d-4a2c-4d50-aada-46c00f5c5dd7") : object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:25.996069 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:25.996035    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:25.996506 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:25.996073    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:25.996506 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:25.996035    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:25.996506 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:25.996146    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:25.996506 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:25.996241    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:25.996506 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:25.996362    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:27.995906 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:27.995864    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:27.996369 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:27.995870    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:27.996369 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:27.995988    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:27.996369 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:27.995866    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:27.996369 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:27.996081    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:27.996369 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:27.996177    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:28.480980 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:28.480900    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:28.481129 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:28.481023    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:28.481129 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:28.481089    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret podName:7093c19d-4a2c-4d50-aada-46c00f5c5dd7 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:36.48107495 +0000 UTC m=+25.097564226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret") pod "global-pull-secret-syncer-fln9v" (UID: "7093c19d-4a2c-4d50-aada-46c00f5c5dd7") : object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:29.689552 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:29.689513    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:29.690007 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:29.689590    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:29.690007 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:29.689708    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:29.690007 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:29.689718    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 02:45:29.690007 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:29.689740    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 02:45:29.690007 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:29.689750    2554 projected.go:194] Error preparing data for projected volume kube-api-access-bxpdl for pod openshift-network-diagnostics/network-check-target-bcmlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:29.690007 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:29.689776    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs podName:fcfae5f4-8103-45fa-beee-bdb243b5ee53 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:45.689757578 +0000 UTC m=+34.306246649 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs") pod "network-metrics-daemon-vnj4x" (UID: "fcfae5f4-8103-45fa-beee-bdb243b5ee53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:29.690007 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:29.689792    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl podName:861af5a6-480a-4494-89c0-0d4416a9fb9f nodeName:}" failed. No retries permitted until 2026-06-02 02:45:45.689785101 +0000 UTC m=+34.306274171 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bxpdl" (UniqueName: "kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl") pod "network-check-target-bcmlx" (UID: "861af5a6-480a-4494-89c0-0d4416a9fb9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:29.995878 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:29.995783    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:29.995878 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:29.995812    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:29.995878 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:29.995843    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:29.996139 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:29.995951    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:29.996139 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:29.996001    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:29.996139 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:29.996098    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:31.996385 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:31.996351    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:31.996913 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:31.996487    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:31.996913 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:31.996487    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:31.996913 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:31.996516    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:31.996913 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:31.996621    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:31.996913 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:31.996760    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:33.113336 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.113051    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vdf4f" event={"ID":"0b560b06-bdcc-4b2b-8cb4-ce91d50a8341","Type":"ContainerStarted","Data":"2686b077423dae5d712a51f6b7c52a6c70d721c2550a8a5b38212ea12da7edf2"}
Jun 02 02:45:33.118070 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.117758    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs" event={"ID":"1120f884-f9f1-41a3-886c-5fe2001bf3f3","Type":"ContainerStarted","Data":"ac9cc0aedd73c765e407d2ca25a38e1ff2373de057c1d62dc28a0c046d3f57f2"}
Jun 02 02:45:33.119570 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.119546    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-msg6m" event={"ID":"adc9d363-18a3-4d49-a07c-e13c52572722","Type":"ContainerStarted","Data":"c4cfb9c6ff008e4d06c29ab3d1166024e9b6b5af93b6b0b4164e3f4c5c2f84b9"}
Jun 02 02:45:33.121766 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.121698    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerStarted","Data":"885e9a8bb7a2b0f4f4450a4e3b554b2d6f5f96734a56a17d94f179d0cfb8faa1"}
Jun 02 02:45:33.126794 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.126594    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6qxdj" event={"ID":"d426a9ef-8544-48f0-9e47-30f550ef29d1","Type":"ContainerStarted","Data":"d8d3c8a49cf01559ae9fb7afbbee3d82e4d3f9b16281b0b4df1837dbf62b96e9"}
Jun 02 02:45:33.128749 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.128638    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-vdf4f" podStartSLOduration=3.101509651 podStartE2EDuration="21.128602162s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:45:14.919656361 +0000 UTC m=+3.536145432" lastFinishedPulling="2026-06-02 02:45:32.946748861 +0000 UTC m=+21.563237943" observedRunningTime="2026-06-02 02:45:33.128543407 +0000 UTC m=+21.745032515" watchObservedRunningTime="2026-06-02 02:45:33.128602162 +0000 UTC m=+21.745091253"
Jun 02 02:45:33.141722 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.141618    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6qxdj" podStartSLOduration=3.160897031 podStartE2EDuration="21.141600661s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:45:14.907245071 +0000 UTC m=+3.523734146" lastFinishedPulling="2026-06-02 02:45:32.887948692 +0000 UTC m=+21.504437776" observedRunningTime="2026-06-02 02:45:33.141301223 +0000 UTC m=+21.757790315" watchObservedRunningTime="2026-06-02 02:45:33.141600661 +0000 UTC m=+21.758089752"
Jun 02 02:45:33.155240 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.155106    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-msg6m" podStartSLOduration=7.156609105 podStartE2EDuration="21.155088326s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:45:14.916047578 +0000 UTC m=+3.532536648" lastFinishedPulling="2026-06-02 02:45:28.914526785 +0000 UTC m=+17.531015869" observedRunningTime="2026-06-02 02:45:33.154643401 +0000 UTC m=+21.771132521" watchObservedRunningTime="2026-06-02 02:45:33.155088326 +0000 UTC m=+21.771577418"
Jun 02 02:45:33.996200 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.996037    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:33.996396 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.996049    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:33.996396 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:33.996050    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:33.996550 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:33.996295    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:33.996797 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:33.996771    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:33.996912 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:33.996885    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:34.096751 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.096722    2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jun 02 02:45:34.130085 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.130054    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs" event={"ID":"1120f884-f9f1-41a3-886c-5fe2001bf3f3","Type":"ContainerStarted","Data":"87b9f614f3e55cb7544f4f2b861dfd0ad8088bbae77e2d636605e910dfa6e980"}
Jun 02 02:45:34.132355 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.132337    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 02:45:34.132664 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.132643    2554 generic.go:358] "Generic (PLEG): container finished" podID="59e50745-ea12-4b4d-9707-0776345bfe9c" containerID="006d4f3a57d4bdb0b56bb8e5da577538859430dc2cdd8247bb3e77d222cfb85a" exitCode=1
Jun 02 02:45:34.132753 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.132704    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerStarted","Data":"79104b797ed521dca57a9562e562742186976773a4190f91da32a48976eba2a7"}
Jun 02 02:45:34.132753 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.132727    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerStarted","Data":"cc23232fb73b11094837e5314745527deaef329eea771db2aa936bc55e2e3d25"}
Jun 02 02:45:34.132753 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.132737    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerStarted","Data":"96d905dd3cbccd1733960f7bcf40f5f4d467c4c87656a423ccb533bc44b4031c"}
Jun 02 02:45:34.132753 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.132745    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerStarted","Data":"28fdbb2f3c1261e13521ed9e341ff18dc3dc0a05d8d065c04b093f133dac9544"}
Jun 02 02:45:34.132753 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.132753    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerDied","Data":"006d4f3a57d4bdb0b56bb8e5da577538859430dc2cdd8247bb3e77d222cfb85a"}
Jun 02 02:45:34.133926 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.133907    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cn8tf" event={"ID":"b170f48b-6b05-4361-90e3-4f5b81d273ea","Type":"ContainerStarted","Data":"0fa3b041263bfd3d5b43ef64019f2ef587e7c522f366d68d9f81c5c0e3db9293"}
Jun 02 02:45:34.135150 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.135124    2554 generic.go:358] "Generic (PLEG): container finished" podID="007548f0-391f-4fc2-8977-615a8a04f403" containerID="5c81bf788f211beae44235f65546bffc7fc91c409afeb1082dab266be495f14b" exitCode=0
Jun 02 02:45:34.135241 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.135191    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4stzd" event={"ID":"007548f0-391f-4fc2-8977-615a8a04f403","Type":"ContainerDied","Data":"5c81bf788f211beae44235f65546bffc7fc91c409afeb1082dab266be495f14b"}
Jun 02 02:45:34.136466 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.136431    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4x78s" event={"ID":"23a084fd-4b0c-4c56-930d-b946a7e1bcbe","Type":"ContainerStarted","Data":"df8fe920c96316619d23ec091cd1fc04cb14719b3c58b87cf2cc579ebfeae495"}
Jun 02 02:45:34.168328 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.168259    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-cn8tf" podStartSLOduration=4.612240736 podStartE2EDuration="22.168248433s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:45:14.923299964 +0000 UTC m=+3.539789047" lastFinishedPulling="2026-06-02 02:45:32.479307667 +0000 UTC m=+21.095796744" observedRunningTime="2026-06-02 02:45:33.189591588 +0000 UTC m=+21.806080679" watchObservedRunningTime="2026-06-02 02:45:34.168248433 +0000 UTC m=+22.784737523"
Jun 02 02:45:34.180083 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.180049    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4x78s" podStartSLOduration=4.622012452 podStartE2EDuration="22.180038447s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:45:14.921306611 +0000 UTC m=+3.537795692" lastFinishedPulling="2026-06-02 02:45:32.479332615 +0000 UTC m=+21.095821687" observedRunningTime="2026-06-02 02:45:34.17973107 +0000 UTC m=+22.796220178" watchObservedRunningTime="2026-06-02 02:45:34.180038447 +0000 UTC m=+22.796527538"
Jun 02 02:45:34.923265 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.923163    2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T02:45:34.096746915Z","UUID":"2bd62fc3-e174-4ddf-a110-cfc59ab5c726","Handler":null,"Name":"","Endpoint":""}
Jun 02 02:45:34.925101 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.925074    2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jun 02 02:45:34.925266 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:34.925122    2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jun 02 02:45:35.140527 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:35.140486    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs" event={"ID":"1120f884-f9f1-41a3-886c-5fe2001bf3f3","Type":"ContainerStarted","Data":"402ca736a0a7d949368e8b5deb6acb011851d5f9c5edb96b106d3cb8e4112279"}
Jun 02 02:45:35.141670 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:35.141649    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jqf8v" event={"ID":"dd9ef910-5486-460d-af31-0785eefbde99","Type":"ContainerStarted","Data":"dd10bce8401c3dba8d1fa2f87a8fe9e274453b84c4da46ab3f8806cf380dd983"}
Jun 02 02:45:35.171695 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:35.171648    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crlcs" podStartSLOduration=3.067822249 podStartE2EDuration="23.171634681s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:45:14.918472769 +0000 UTC m=+3.534961845" lastFinishedPulling="2026-06-02 02:45:35.022285196 +0000 UTC m=+23.638774277" observedRunningTime="2026-06-02 02:45:35.156803248 +0000 UTC m=+23.773292349" watchObservedRunningTime="2026-06-02 02:45:35.171634681 +0000 UTC m=+23.788123772"
Jun 02 02:45:35.172013 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:35.171987    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jqf8v" podStartSLOduration=5.202125443 podStartE2EDuration="23.171978818s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:45:14.916261686 +0000 UTC m=+3.532750765" lastFinishedPulling="2026-06-02 02:45:32.886115069 +0000 UTC m=+21.502604140" observedRunningTime="2026-06-02 02:45:35.17131367 +0000 UTC m=+23.787802774" watchObservedRunningTime="2026-06-02 02:45:35.171978818 +0000 UTC m=+23.788467909"
Jun 02 02:45:35.995997 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:35.995810    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:35.996188 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:35.995810    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:35.996188 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:35.996072    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:35.996304 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:35.996196    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:35.996304 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:35.995820    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:35.996399 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:35.996312    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:36.146061 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:36.146031    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 02:45:36.146522 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:36.146405    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerStarted","Data":"4208927dba46aa3d651994622f53fd49652ee437ccefb1b3efbc9bc076cd2de5"}
Jun 02 02:45:36.224256 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:36.224215    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:36.225258 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:36.225232    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:36.534951 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:36.534923    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:36.535153 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:36.535088    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:36.535236 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:36.535151    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret podName:7093c19d-4a2c-4d50-aada-46c00f5c5dd7 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:52.535137087 +0000 UTC m=+41.151626156 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret") pod "global-pull-secret-syncer-fln9v" (UID: "7093c19d-4a2c-4d50-aada-46c00f5c5dd7") : object "kube-system"/"original-pull-secret" not registered
Jun 02 02:45:37.148436 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:37.148411    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:37.148864 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:37.148834    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-msg6m"
Jun 02 02:45:37.995528 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:37.995317    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:37.995711 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:37.995317    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:37.995711 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:37.995636    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:37.995819 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:37.995714    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:37.995819 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:37.995334    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:37.995882 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:37.995857    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:38.153440 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:38.153409    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 02:45:38.154235 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:38.153806    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerStarted","Data":"752fcd16a68997e47a47bad13ddfc39c85bb09895de75c9e93028e41028fbd9a"}
Jun 02 02:45:38.154235 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:38.154067    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:38.154235 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:38.154094    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:38.154235 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:38.154221    2554 scope.go:117] "RemoveContainer" containerID="006d4f3a57d4bdb0b56bb8e5da577538859430dc2cdd8247bb3e77d222cfb85a"
Jun 02 02:45:38.171068 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:38.171046    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:39.495122 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:39.495097    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fln9v"]
Jun 02 02:45:39.495416 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:39.495213    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:39.495416 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:39.495295    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:39.497965 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:39.497904    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bcmlx"]
Jun 02 02:45:39.498089 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:39.498036    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:39.498147 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:39.498131    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:39.498554 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:39.498531    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vnj4x"]
Jun 02 02:45:39.498655 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:39.498637    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:39.498766 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:39.498746    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:40.160804 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.160737    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 02:45:40.161151 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.161114    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" event={"ID":"59e50745-ea12-4b4d-9707-0776345bfe9c","Type":"ContainerStarted","Data":"6baaf3363a4d92ad9309ea69770e5e497c9b4b343483f86841b9751c31130ebd"}
Jun 02 02:45:40.161337 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.161316    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:40.162652 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.162627    2554 generic.go:358] "Generic (PLEG): container finished" podID="007548f0-391f-4fc2-8977-615a8a04f403" containerID="ebff27d1925c2f95e11ffe9ca791eb8218250a688d6028a8c669227af53595b0" exitCode=0
Jun 02 02:45:40.162731 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.162667    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4stzd" event={"ID":"007548f0-391f-4fc2-8977-615a8a04f403","Type":"ContainerDied","Data":"ebff27d1925c2f95e11ffe9ca791eb8218250a688d6028a8c669227af53595b0"}
Jun 02 02:45:40.176157 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.176140    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:45:40.188385 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.188352    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" podStartSLOduration=10.2146777 podStartE2EDuration="28.18833964s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:45:14.91487222 +0000 UTC m=+3.531361291" lastFinishedPulling="2026-06-02 02:45:32.888534149 +0000 UTC m=+21.505023231" observedRunningTime="2026-06-02 02:45:40.188340612 +0000 UTC m=+28.804829686" watchObservedRunningTime="2026-06-02 02:45:40.18833964 +0000 UTC m=+28.804828725"
Jun 02 02:45:40.995320 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.995290    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:40.995820 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.995327    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:40.995820 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:40.995407    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:40.995820 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:40.995440    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:40.995820 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:40.995517    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:40.995820 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:40.995586    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:42.168190 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:42.168161    2554 generic.go:358] "Generic (PLEG): container finished" podID="007548f0-391f-4fc2-8977-615a8a04f403" containerID="eecee7f8c70ccc1c74f90c6cdfbe3c3db3d09eab51e212b438b9ec1f31242101" exitCode=0
Jun 02 02:45:42.168595 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:42.168240    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4stzd" event={"ID":"007548f0-391f-4fc2-8977-615a8a04f403","Type":"ContainerDied","Data":"eecee7f8c70ccc1c74f90c6cdfbe3c3db3d09eab51e212b438b9ec1f31242101"}
Jun 02 02:45:42.182723 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:42.182539    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp" podUID="59e50745-ea12-4b4d-9707-0776345bfe9c" containerName="ovnkube-controller" probeResult="failure" output=""
Jun 02 02:45:42.996190 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:42.996158    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:42.996190 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:42.996173    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:42.996437 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:42.996158    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:42.996437 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:42.996257    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:42.996437 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:42.996353    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:42.996437 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:42.996392    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:44.173663 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:44.173631    2554 generic.go:358] "Generic (PLEG): container finished" podID="007548f0-391f-4fc2-8977-615a8a04f403" containerID="a12517a9ad7d6f13c2b1c38b541bdbb0137eec2bf6b0f8fabf017c878e940ad2" exitCode=0
Jun 02 02:45:44.174173 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:44.173680    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4stzd" event={"ID":"007548f0-391f-4fc2-8977-615a8a04f403","Type":"ContainerDied","Data":"a12517a9ad7d6f13c2b1c38b541bdbb0137eec2bf6b0f8fabf017c878e940ad2"}
Jun 02 02:45:44.996392 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:44.996355    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:44.996392 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:44.996389    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:44.996820 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:44.996355    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:44.996820 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:44.996519    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:45:44.996820 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:44.996657    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcmlx" podUID="861af5a6-480a-4494-89c0-0d4416a9fb9f"
Jun 02 02:45:44.996820 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:44.996771    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fln9v" podUID="7093c19d-4a2c-4d50-aada-46c00f5c5dd7"
Jun 02 02:45:45.662663 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.662585    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeReady"
Jun 02 02:45:45.663150 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.662747    2554 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
Jun 02 02:45:45.707123 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.707089    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:45.707301 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.707206    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:45.707301 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:45.707254    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:45.707416 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:45.707322    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 02:45:45.707416 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:45.707337    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 02:45:45.707416 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:45.707349    2554 projected.go:194] Error preparing data for projected volume kube-api-access-bxpdl for pod openshift-network-diagnostics/network-check-target-bcmlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:45.707416 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:45.707337    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs podName:fcfae5f4-8103-45fa-beee-bdb243b5ee53 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:17.707312162 +0000 UTC m=+66.323801250 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs") pod "network-metrics-daemon-vnj4x" (UID: "fcfae5f4-8103-45fa-beee-bdb243b5ee53") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 02:45:45.707416 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:45.707400    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl podName:861af5a6-480a-4494-89c0-0d4416a9fb9f nodeName:}" failed. No retries permitted until 2026-06-02 02:46:17.707388957 +0000 UTC m=+66.323878029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bxpdl" (UniqueName: "kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl") pod "network-check-target-bcmlx" (UID: "861af5a6-480a-4494-89c0-0d4416a9fb9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 02:45:45.707802 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.707680    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"]
Jun 02 02:45:45.726501 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.726472    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55d64b65bb-ndbb5"]
Jun 02 02:45:45.726714 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.726694    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:45.730061 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.730033    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jun 02 02:45:45.730184 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.730067    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vmqqx\""
Jun 02 02:45:45.730184 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.730074    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jun 02 02:45:45.746530 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.746508    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"]
Jun 02 02:45:45.746792 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.746771    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:45.749747 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.749728    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jun 02 02:45:45.750087 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.749957    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jun 02 02:45:45.750087 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.749959    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jun 02 02:45:45.750087 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.750030    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bqz7l\""
Jun 02 02:45:45.755336 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.755314    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jun 02 02:45:45.773127 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.773101    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"]
Jun 02 02:45:45.773270 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.773251    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:45.776503 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.776444    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jun 02 02:45:45.776616 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.776519    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jun 02 02:45:45.776736 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.776714    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jun 02 02:45:45.777098 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.777079    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jun 02 02:45:45.777179 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.777079    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jun 02 02:45:45.777292 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.777262    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jun 02 02:45:45.777418 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.777390    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jun 02 02:45:45.791022 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.791000    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"]
Jun 02 02:45:45.791173 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.791154    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"
Jun 02 02:45:45.793921 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.793900    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-56spg\""
Jun 02 02:45:45.794035 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.793944    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jun 02 02:45:45.808132 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.808105    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:45.808241 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.808144    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:45.814903 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.814882    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"]
Jun 02 02:45:45.815031 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.814909    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9x5hx"]
Jun 02 02:45:45.815092 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.815055    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:45.818316 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.818294    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jun 02 02:45:45.834029 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.834009    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"]
Jun 02 02:45:45.834158 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.834036    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55d64b65bb-ndbb5"]
Jun 02 02:45:45.834158 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.834050    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"]
Jun 02 02:45:45.834158 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.834063    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nhglj"]
Jun 02 02:45:45.834268 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.834175    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:45.840146 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.837166    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jun 02 02:45:45.840246 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.837119    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jun 02 02:45:45.840376 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.840304    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5469w\""
Jun 02 02:45:45.859518 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.859493    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"]
Jun 02 02:45:45.859518 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.859521    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9x5hx"]
Jun 02 02:45:45.859687 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.859531    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nhglj"]
Jun 02 02:45:45.859687 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.859554    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:45.862057 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.862038    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jun 02 02:45:45.862181 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.862162    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kcfjp\""
Jun 02 02:45:45.862263 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.862246    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jun 02 02:45:45.862320 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.862270    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jun 02 02:45:45.909186 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909149    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:45.909372 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909191    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:45.909372 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909219    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb3655a7-5a4c-44fe-abd9-927b6b30adaf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74879db984-l9fkf\" (UID: \"bb3655a7-5a4c-44fe-abd9-927b6b30adaf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"
Jun 02 02:45:45.909372 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909249    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:45.909372 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909271    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zthf\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-kube-api-access-6zthf\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:45.909372 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:45.909312    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 02 02:45:45.909372 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909339    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-image-registry-private-configuration\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:45.909672 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:45.909382    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert podName:e3d7f74d-20a3-484c-b9ca-4ab73b6a292c nodeName:}" failed. No retries permitted until 2026-06-02 02:45:46.409360557 +0000 UTC m=+35.025849645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-m6jzt" (UID: "e3d7f74d-20a3-484c-b9ca-4ab73b6a292c") : secret "networking-console-plugin-cert" not found
Jun 02 02:45:45.909672 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909410    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-ca-trust-extracted\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:45.909672 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909443    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-trusted-ca\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:45.909672 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909487    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/ec92eb63-9f2a-468a-aafc-e57f6fdec868-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:45.909672 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909520    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-bound-sa-token\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:45.909672 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909546    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/317556c1-8dc7-48a5-aac4-31e57c41b6fe-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d65cd667c-4wsnk\" (UID: \"317556c1-8dc7-48a5-aac4-31e57c41b6fe\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:45.909672 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909586    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-ca\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:45.909672 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909627    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:45.909672 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909656    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8z46\" (UniqueName: \"kubernetes.io/projected/ec92eb63-9f2a-468a-aafc-e57f6fdec868-kube-api-access-q8z46\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:45.910031 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909685    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84n2d\" (UniqueName: \"kubernetes.io/projected/bb3655a7-5a4c-44fe-abd9-927b6b30adaf-kube-api-access-84n2d\") pod \"managed-serviceaccount-addon-agent-74879db984-l9fkf\" (UID: \"bb3655a7-5a4c-44fe-abd9-927b6b30adaf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"
Jun 02 02:45:45.910031 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909727    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-hub\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:45.910031 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909789    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-certificates\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:45.910031 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909816    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-installation-pull-secrets\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:45.910031 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909861    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8b68\" (UniqueName: \"kubernetes.io/projected/317556c1-8dc7-48a5-aac4-31e57c41b6fe-kube-api-access-p8b68\") pod \"klusterlet-addon-workmgr-5d65cd667c-4wsnk\" (UID: \"317556c1-8dc7-48a5-aac4-31e57c41b6fe\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:45.910031 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909909    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/317556c1-8dc7-48a5-aac4-31e57c41b6fe-tmp\") pod \"klusterlet-addon-workmgr-5d65cd667c-4wsnk\" (UID: \"317556c1-8dc7-48a5-aac4-31e57c41b6fe\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:45.910031 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.909941    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:45.910293 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:45.910264    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:46.011129 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011045    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-hub\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.011129 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011088    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-installation-pull-secrets\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.011129 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011113    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p8b68\" (UniqueName: \"kubernetes.io/projected/317556c1-8dc7-48a5-aac4-31e57c41b6fe-kube-api-access-p8b68\") pod \"klusterlet-addon-workmgr-5d65cd667c-4wsnk\" (UID: \"317556c1-8dc7-48a5-aac4-31e57c41b6fe\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:46.011398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011143    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:46.011398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011189    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.011398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011213    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.011398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011240    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.011398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011261    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db50fd2b-9b3f-403f-a131-bf4a898ca513-tmp-dir\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.011398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011282    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-84n2d\" (UniqueName: \"kubernetes.io/projected/bb3655a7-5a4c-44fe-abd9-927b6b30adaf-kube-api-access-84n2d\") pod \"managed-serviceaccount-addon-agent-74879db984-l9fkf\" (UID: \"bb3655a7-5a4c-44fe-abd9-927b6b30adaf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"
Jun 02 02:45:46.011398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011312    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/317556c1-8dc7-48a5-aac4-31e57c41b6fe-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d65cd667c-4wsnk\" (UID: \"317556c1-8dc7-48a5-aac4-31e57c41b6fe\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:46.011398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011341    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q8z46\" (UniqueName: \"kubernetes.io/projected/ec92eb63-9f2a-468a-aafc-e57f6fdec868-kube-api-access-q8z46\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011406    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-certificates\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011443    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8hj6\" (UniqueName: \"kubernetes.io/projected/db50fd2b-9b3f-403f-a131-bf4a898ca513-kube-api-access-l8hj6\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011504    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/317556c1-8dc7-48a5-aac4-31e57c41b6fe-tmp\") pod \"klusterlet-addon-workmgr-5d65cd667c-4wsnk\" (UID: \"317556c1-8dc7-48a5-aac4-31e57c41b6fe\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011531    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4fhv\" (UniqueName: \"kubernetes.io/projected/a457d44f-df4a-4ef5-9de4-632e3318feee-kube-api-access-t4fhv\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011559    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011609    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb3655a7-5a4c-44fe-abd9-927b6b30adaf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74879db984-l9fkf\" (UID: \"bb3655a7-5a4c-44fe-abd9-927b6b30adaf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011651    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6zthf\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-kube-api-access-6zthf\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.011658    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.011675    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d64b65bb-ndbb5: secret "image-registry-tls" not found
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011680    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-ca-trust-extracted\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011707    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/ec92eb63-9f2a-468a-aafc-e57f6fdec868-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.011727    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls podName:6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:46.511712026 +0000 UTC m=+35.128201291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls") pod "image-registry-55d64b65bb-ndbb5" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48") : secret "image-registry-tls" not found
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011750    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-image-registry-private-configuration\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011771    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-trusted-ca\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.011823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011825    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-bound-sa-token\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.012570 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011866    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-ca\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.012570 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.011903    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db50fd2b-9b3f-403f-a131-bf4a898ca513-config-volume\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.012570 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.012272    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/317556c1-8dc7-48a5-aac4-31e57c41b6fe-tmp\") pod \"klusterlet-addon-workmgr-5d65cd667c-4wsnk\" (UID: \"317556c1-8dc7-48a5-aac4-31e57c41b6fe\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:46.012570 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.012428    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/ec92eb63-9f2a-468a-aafc-e57f6fdec868-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.012916 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.012869    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-trusted-ca\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.013311 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.013264    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-certificates\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.014084 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.014056    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-ca-trust-extracted\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.016748 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.016723    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-hub\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.016963 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.016932    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-image-registry-private-configuration\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.017111 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.017090    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-installation-pull-secrets\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.017180 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.017148    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb3655a7-5a4c-44fe-abd9-927b6b30adaf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74879db984-l9fkf\" (UID: \"bb3655a7-5a4c-44fe-abd9-927b6b30adaf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"
Jun 02 02:45:46.017239 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.017207    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.017291 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.017239    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.017394 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.017380    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ec92eb63-9f2a-468a-aafc-e57f6fdec868-ca\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.017972 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.017931    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/317556c1-8dc7-48a5-aac4-31e57c41b6fe-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d65cd667c-4wsnk\" (UID: \"317556c1-8dc7-48a5-aac4-31e57c41b6fe\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:46.021300 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.021232    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8z46\" (UniqueName: \"kubernetes.io/projected/ec92eb63-9f2a-468a-aafc-e57f6fdec868-kube-api-access-q8z46\") pod \"cluster-proxy-proxy-agent-6865c5b45c-gmb6j\" (UID: \"ec92eb63-9f2a-468a-aafc-e57f6fdec868\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.021996 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.021970    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8b68\" (UniqueName: \"kubernetes.io/projected/317556c1-8dc7-48a5-aac4-31e57c41b6fe-kube-api-access-p8b68\") pod \"klusterlet-addon-workmgr-5d65cd667c-4wsnk\" (UID: \"317556c1-8dc7-48a5-aac4-31e57c41b6fe\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:46.022445 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.022399    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-bound-sa-token\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.023027 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.023006    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zthf\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-kube-api-access-6zthf\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.023594 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.023575    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84n2d\" (UniqueName: \"kubernetes.io/projected/bb3655a7-5a4c-44fe-abd9-927b6b30adaf-kube-api-access-84n2d\") pod \"managed-serviceaccount-addon-agent-74879db984-l9fkf\" (UID: \"bb3655a7-5a4c-44fe-abd9-927b6b30adaf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"
Jun 02 02:45:46.092809 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.092774    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:45:46.101928 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.101902    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"
Jun 02 02:45:46.112968 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.112936    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t4fhv\" (UniqueName: \"kubernetes.io/projected/a457d44f-df4a-4ef5-9de4-632e3318feee-kube-api-access-t4fhv\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:46.113089 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.113032    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db50fd2b-9b3f-403f-a131-bf4a898ca513-config-volume\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.113089 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.113068    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:46.113202 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.113118    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.113255 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.113236    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 02:45:46.113321 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.113306    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls podName:db50fd2b-9b3f-403f-a131-bf4a898ca513 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:46.613287216 +0000 UTC m=+35.229776298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls") pod "dns-default-9x5hx" (UID: "db50fd2b-9b3f-403f-a131-bf4a898ca513") : secret "dns-default-metrics-tls" not found
Jun 02 02:45:46.113613 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.113589    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db50fd2b-9b3f-403f-a131-bf4a898ca513-tmp-dir\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.113693 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.113660    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l8hj6\" (UniqueName: \"kubernetes.io/projected/db50fd2b-9b3f-403f-a131-bf4a898ca513-kube-api-access-l8hj6\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.113999 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.113959    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db50fd2b-9b3f-403f-a131-bf4a898ca513-config-volume\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.114108 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.114064    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 02:45:46.114211 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.114111    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert podName:a457d44f-df4a-4ef5-9de4-632e3318feee nodeName:}" failed. No retries permitted until 2026-06-02 02:45:46.614094247 +0000 UTC m=+35.230583331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert") pod "ingress-canary-nhglj" (UID: "a457d44f-df4a-4ef5-9de4-632e3318feee") : secret "canary-serving-cert" not found
Jun 02 02:45:46.114211 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.114183    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db50fd2b-9b3f-403f-a131-bf4a898ca513-tmp-dir\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.123123 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.123100    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4fhv\" (UniqueName: \"kubernetes.io/projected/a457d44f-df4a-4ef5-9de4-632e3318feee-kube-api-access-t4fhv\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:46.123219 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.123099    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8hj6\" (UniqueName: \"kubernetes.io/projected/db50fd2b-9b3f-403f-a131-bf4a898ca513-kube-api-access-l8hj6\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.137055 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.137029    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:46.289589 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.289261    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"]
Jun 02 02:45:46.290177 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.290148    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf"]
Jun 02 02:45:46.295888 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.295731    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"]
Jun 02 02:45:46.299194 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:46.299164    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod317556c1_8dc7_48a5_aac4_31e57c41b6fe.slice/crio-d31fd7dc31f31c655525d81a75c225706d40462c9b8c488b77d0b2d104d6b1aa WatchSource:0}: Error finding container d31fd7dc31f31c655525d81a75c225706d40462c9b8c488b77d0b2d104d6b1aa: Status 404 returned error can't find the container with id d31fd7dc31f31c655525d81a75c225706d40462c9b8c488b77d0b2d104d6b1aa
Jun 02 02:45:46.415884 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.415851    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:46.416118 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.416028    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 02 02:45:46.416118 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.416117    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert podName:e3d7f74d-20a3-484c-b9ca-4ab73b6a292c nodeName:}" failed. No retries permitted until 2026-06-02 02:45:47.416095598 +0000 UTC m=+36.032584671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-m6jzt" (UID: "e3d7f74d-20a3-484c-b9ca-4ab73b6a292c") : secret "networking-console-plugin-cert" not found
Jun 02 02:45:46.517352 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.517315    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:46.517567 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.517511    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 02:45:46.517567 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.517532    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d64b65bb-ndbb5: secret "image-registry-tls" not found
Jun 02 02:45:46.517702 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.517604    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls podName:6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:47.517582844 +0000 UTC m=+36.134071923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls") pod "image-registry-55d64b65bb-ndbb5" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48") : secret "image-registry-tls" not found
Jun 02 02:45:46.618728 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.618643    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:46.618728 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.618720    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:46.618951 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.618790    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 02:45:46.618951 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.618865    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert podName:a457d44f-df4a-4ef5-9de4-632e3318feee nodeName:}" failed. No retries permitted until 2026-06-02 02:45:47.61884547 +0000 UTC m=+36.235334547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert") pod "ingress-canary-nhglj" (UID: "a457d44f-df4a-4ef5-9de4-632e3318feee") : secret "canary-serving-cert" not found
Jun 02 02:45:46.618951 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.618890    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 02:45:46.618951 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:46.618947    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls podName:db50fd2b-9b3f-403f-a131-bf4a898ca513 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:47.618930496 +0000 UTC m=+36.235419579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls") pod "dns-default-9x5hx" (UID: "db50fd2b-9b3f-403f-a131-bf4a898ca513") : secret "dns-default-metrics-tls" not found
Jun 02 02:45:46.995741 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.995664    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:46.995741 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.995730    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:45:46.996311 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.995744    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:45:47.000398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:46.999780    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgk2p\""
Jun 02 02:45:47.000398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.000023    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ldjq9\""
Jun 02 02:45:47.000398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.000060    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 02 02:45:47.000398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.000224    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 02 02:45:47.000398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.000261    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 02 02:45:47.000398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.000405    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 02 02:45:47.190962 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.190899    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf" event={"ID":"bb3655a7-5a4c-44fe-abd9-927b6b30adaf","Type":"ContainerStarted","Data":"d946b0e81d84f5d86ca30f4d5d1850152eebc50a58da0aad38a9631c9f6317f8"}
Jun 02 02:45:47.192972 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.192896    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" event={"ID":"ec92eb63-9f2a-468a-aafc-e57f6fdec868","Type":"ContainerStarted","Data":"3434ef554699b2690d7c9b7d4375b98fffcc8c21687dcc279128102b2ae5bcaa"}
Jun 02 02:45:47.195245 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.195186    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk" event={"ID":"317556c1-8dc7-48a5-aac4-31e57c41b6fe","Type":"ContainerStarted","Data":"d31fd7dc31f31c655525d81a75c225706d40462c9b8c488b77d0b2d104d6b1aa"}
Jun 02 02:45:47.425836 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.425675    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:47.426029 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:47.425852    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 02 02:45:47.426029 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:47.425921    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert podName:e3d7f74d-20a3-484c-b9ca-4ab73b6a292c nodeName:}" failed. No retries permitted until 2026-06-02 02:45:49.425902526 +0000 UTC m=+38.042391595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-m6jzt" (UID: "e3d7f74d-20a3-484c-b9ca-4ab73b6a292c") : secret "networking-console-plugin-cert" not found
Jun 02 02:45:47.526837 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.526795    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:47.527048 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:47.527028    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 02:45:47.527116 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:47.527053    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d64b65bb-ndbb5: secret "image-registry-tls" not found
Jun 02 02:45:47.527164 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:47.527121    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls podName:6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:49.527100286 +0000 UTC m=+38.143589372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls") pod "image-registry-55d64b65bb-ndbb5" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48") : secret "image-registry-tls" not found
Jun 02 02:45:47.628165 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.628071    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:47.628354 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:47.628220    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:47.628422 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:47.628375    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 02:45:47.628506 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:47.628487    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert podName:a457d44f-df4a-4ef5-9de4-632e3318feee nodeName:}" failed. No retries permitted until 2026-06-02 02:45:49.628467106 +0000 UTC m=+38.244956189 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert") pod "ingress-canary-nhglj" (UID: "a457d44f-df4a-4ef5-9de4-632e3318feee") : secret "canary-serving-cert" not found
Jun 02 02:45:47.628799 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:47.628702    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 02:45:47.628799 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:47.628769    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls podName:db50fd2b-9b3f-403f-a131-bf4a898ca513 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:49.628750946 +0000 UTC m=+38.245240018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls") pod "dns-default-9x5hx" (UID: "db50fd2b-9b3f-403f-a131-bf4a898ca513") : secret "dns-default-metrics-tls" not found
Jun 02 02:45:49.452215 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:49.452126    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:49.452651 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:49.452292    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 02 02:45:49.452651 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:49.452368    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert podName:e3d7f74d-20a3-484c-b9ca-4ab73b6a292c nodeName:}" failed. No retries permitted until 2026-06-02 02:45:53.452348669 +0000 UTC m=+42.068837748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-m6jzt" (UID: "e3d7f74d-20a3-484c-b9ca-4ab73b6a292c") : secret "networking-console-plugin-cert" not found
Jun 02 02:45:49.553233 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:49.553183    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:49.553401 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:49.553337    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 02:45:49.553401 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:49.553358    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d64b65bb-ndbb5: secret "image-registry-tls" not found
Jun 02 02:45:49.553493 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:49.553435    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls podName:6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:53.553413171 +0000 UTC m=+42.169902245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls") pod "image-registry-55d64b65bb-ndbb5" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48") : secret "image-registry-tls" not found
Jun 02 02:45:49.654006 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:49.653973    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:49.654151 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:49.654021    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:49.654151 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:49.654122    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 02:45:49.654273 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:49.654186    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert podName:a457d44f-df4a-4ef5-9de4-632e3318feee nodeName:}" failed. No retries permitted until 2026-06-02 02:45:53.654171627 +0000 UTC m=+42.270660696 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert") pod "ingress-canary-nhglj" (UID: "a457d44f-df4a-4ef5-9de4-632e3318feee") : secret "canary-serving-cert" not found
Jun 02 02:45:49.654273 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:49.654131    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 02:45:49.654372 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:49.654284    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls podName:db50fd2b-9b3f-403f-a131-bf4a898ca513 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:53.654267407 +0000 UTC m=+42.270756487 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls") pod "dns-default-9x5hx" (UID: "db50fd2b-9b3f-403f-a131-bf4a898ca513") : secret "dns-default-metrics-tls" not found
Jun 02 02:45:52.578800 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:52.578768    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:52.582510 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:52.582484    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7093c19d-4a2c-4d50-aada-46c00f5c5dd7-original-pull-secret\") pod \"global-pull-secret-syncer-fln9v\" (UID: \"7093c19d-4a2c-4d50-aada-46c00f5c5dd7\") " pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:52.713035 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:52.713001    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fln9v"
Jun 02 02:45:53.487819 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:53.487777    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:45:53.488008 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:53.487945    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 02 02:45:53.488058 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:53.488017    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert podName:e3d7f74d-20a3-484c-b9ca-4ab73b6a292c nodeName:}" failed. No retries permitted until 2026-06-02 02:46:01.487999856 +0000 UTC m=+50.104488925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-m6jzt" (UID: "e3d7f74d-20a3-484c-b9ca-4ab73b6a292c") : secret "networking-console-plugin-cert" not found
Jun 02 02:45:53.588440 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:53.588400    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:45:53.588878 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:53.588584    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 02:45:53.588878 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:53.588601    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d64b65bb-ndbb5: secret "image-registry-tls" not found
Jun 02 02:45:53.588878 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:53.588674    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls podName:6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:01.588652711 +0000 UTC m=+50.205141781 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls") pod "image-registry-55d64b65bb-ndbb5" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48") : secret "image-registry-tls" not found
Jun 02 02:45:53.689474 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:53.689422    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:45:53.689649 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:53.689577    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:45:53.689649 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:53.689577    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 02:45:53.689649 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:53.689637    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 02:45:53.689818 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:53.689681    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls podName:db50fd2b-9b3f-403f-a131-bf4a898ca513 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:01.689662912 +0000 UTC m=+50.306151997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls") pod "dns-default-9x5hx" (UID: "db50fd2b-9b3f-403f-a131-bf4a898ca513") : secret "dns-default-metrics-tls" not found
Jun 02 02:45:53.689818 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:45:53.689699    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert podName:a457d44f-df4a-4ef5-9de4-632e3318feee nodeName:}" failed. No retries permitted until 2026-06-02 02:46:01.689690301 +0000 UTC m=+50.306179370 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert") pod "ingress-canary-nhglj" (UID: "a457d44f-df4a-4ef5-9de4-632e3318feee") : secret "canary-serving-cert" not found
Jun 02 02:45:56.162824 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.162794    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fln9v"]
Jun 02 02:45:56.168188 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:45:56.168153    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7093c19d_4a2c_4d50_aada_46c00f5c5dd7.slice/crio-c0bcf77696f56e00424e60830b100226f3ed7156561a394564c3194592899dbb WatchSource:0}: Error finding container c0bcf77696f56e00424e60830b100226f3ed7156561a394564c3194592899dbb: Status 404 returned error can't find the container with id c0bcf77696f56e00424e60830b100226f3ed7156561a394564c3194592899dbb
Jun 02 02:45:56.217946 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.217910    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4stzd" event={"ID":"007548f0-391f-4fc2-8977-615a8a04f403","Type":"ContainerStarted","Data":"acbe82aef3ce0372041483bdf94be5545f622bf0e68a99e184eb8062f88cf9cf"}
Jun 02 02:45:56.219030 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.219004    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fln9v" event={"ID":"7093c19d-4a2c-4d50-aada-46c00f5c5dd7","Type":"ContainerStarted","Data":"c0bcf77696f56e00424e60830b100226f3ed7156561a394564c3194592899dbb"}
Jun 02 02:45:56.220310 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.220278    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk" event={"ID":"317556c1-8dc7-48a5-aac4-31e57c41b6fe","Type":"ContainerStarted","Data":"7f71ea8773f589fb39f02cd7172744bb3512f9a65b44565f5d4029bf6de7fb00"}
Jun 02 02:45:56.220497 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.220474    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:56.221696 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.221675    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" event={"ID":"ec92eb63-9f2a-468a-aafc-e57f6fdec868","Type":"ContainerStarted","Data":"b04eff31ccbe3d319331894c274942d9d17ae407f84c972183fcc06fce2d2e5f"}
Jun 02 02:45:56.222535 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.222516    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:45:56.223108 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.223074    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf" event={"ID":"bb3655a7-5a4c-44fe-abd9-927b6b30adaf","Type":"ContainerStarted","Data":"37ee212c5324679670ef68abf327a5fc36febe0ae5cc437cf2d827dceb4157ad"}
Jun 02 02:45:56.255085 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.254994    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf" podStartSLOduration=3.574202879 podStartE2EDuration="13.25498037s" podCreationTimestamp="2026-06-02 02:45:43 +0000 UTC" firstStartedPulling="2026-06-02 02:45:46.298013991 +0000 UTC m=+34.914503061" lastFinishedPulling="2026-06-02 02:45:55.978791469 +0000 UTC m=+44.595280552" observedRunningTime="2026-06-02 02:45:56.254594928 +0000 UTC m=+44.871084021" watchObservedRunningTime="2026-06-02 02:45:56.25498037 +0000 UTC m=+44.871469460"
Jun 02 02:45:56.272543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:56.272502    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk" podStartSLOduration=3.539426952 podStartE2EDuration="13.272480377s" podCreationTimestamp="2026-06-02 02:45:43 +0000 UTC" firstStartedPulling="2026-06-02 02:45:46.301915706 +0000 UTC m=+34.918404783" lastFinishedPulling="2026-06-02 02:45:56.034969117 +0000 UTC m=+44.651458208" observedRunningTime="2026-06-02 02:45:56.272305676 +0000 UTC m=+44.888794767" watchObservedRunningTime="2026-06-02 02:45:56.272480377 +0000 UTC m=+44.888969467"
Jun 02 02:45:57.229068 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:57.229031    2554 generic.go:358] "Generic (PLEG): container finished" podID="007548f0-391f-4fc2-8977-615a8a04f403" containerID="acbe82aef3ce0372041483bdf94be5545f622bf0e68a99e184eb8062f88cf9cf" exitCode=0
Jun 02 02:45:57.229498 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:57.229117    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4stzd" event={"ID":"007548f0-391f-4fc2-8977-615a8a04f403","Type":"ContainerDied","Data":"acbe82aef3ce0372041483bdf94be5545f622bf0e68a99e184eb8062f88cf9cf"}
Jun 02 02:45:58.234842 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:58.234807    2554 generic.go:358] "Generic (PLEG): container finished" podID="007548f0-391f-4fc2-8977-615a8a04f403" containerID="9dbf742732709186831f42ef3ec447b9a5e4d069b91454a1fd8ab84a76b74728" exitCode=0
Jun 02 02:45:58.235282 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:58.234907    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4stzd" event={"ID":"007548f0-391f-4fc2-8977-615a8a04f403","Type":"ContainerDied","Data":"9dbf742732709186831f42ef3ec447b9a5e4d069b91454a1fd8ab84a76b74728"}
Jun 02 02:45:59.240956 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:59.240925    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4stzd" event={"ID":"007548f0-391f-4fc2-8977-615a8a04f403","Type":"ContainerStarted","Data":"221fcbd6266ee85060fb187eb45751795fd455ce845be67d269a7ac3bc9f9099"}
Jun 02 02:45:59.243256 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:59.243231    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" event={"ID":"ec92eb63-9f2a-468a-aafc-e57f6fdec868","Type":"ContainerStarted","Data":"aff3809d85d078c594733ace6c02674a6bff580401d3ca280454c6c0c5b8464e"}
Jun 02 02:45:59.243356 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:59.243263    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" event={"ID":"ec92eb63-9f2a-468a-aafc-e57f6fdec868","Type":"ContainerStarted","Data":"ace9410151db8c36e27829644fe85c5fdc3c0327ff83db4f6038d4bfaa8b2389"}
Jun 02 02:45:59.265466 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:59.265396    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4stzd" podStartSLOduration=6.162363006 podStartE2EDuration="47.265381461s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:45:14.911930677 +0000 UTC m=+3.528419750" lastFinishedPulling="2026-06-02 02:45:56.014949121 +0000 UTC m=+44.631438205" observedRunningTime="2026-06-02 02:45:59.263377455 +0000 UTC m=+47.879866546" watchObservedRunningTime="2026-06-02 02:45:59.265381461 +0000 UTC m=+47.881870552"
Jun 02 02:45:59.282512 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:45:59.282447    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" podStartSLOduration=3.94070664 podStartE2EDuration="16.282435362s" podCreationTimestamp="2026-06-02 02:45:43 +0000 UTC" firstStartedPulling="2026-06-02 02:45:46.297410823 +0000 UTC m=+34.913899892" lastFinishedPulling="2026-06-02 02:45:58.639139545 +0000 UTC m=+47.255628614" observedRunningTime="2026-06-02 02:45:59.281728527 +0000 UTC m=+47.898217620" watchObservedRunningTime="2026-06-02 02:45:59.282435362 +0000 UTC m=+47.898924454"
Jun 02 02:46:01.249706 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:01.249613    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fln9v" event={"ID":"7093c19d-4a2c-4d50-aada-46c00f5c5dd7","Type":"ContainerStarted","Data":"609c8c85979231cd9367324d5589f2d7d5f48d54b618f8f8e455a7cf820eaf06"}
Jun 02 02:46:01.264630 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:01.264572    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fln9v" podStartSLOduration=36.585285137 podStartE2EDuration="41.264557594s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:56.170203853 +0000 UTC m=+44.786692924" lastFinishedPulling="2026-06-02 02:46:00.849476294 +0000 UTC m=+49.465965381" observedRunningTime="2026-06-02 02:46:01.263936726 +0000 UTC m=+49.880425814" watchObservedRunningTime="2026-06-02 02:46:01.264557594 +0000 UTC m=+49.881046686"
Jun 02 02:46:01.560300 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:01.560267    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:46:01.560479 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:01.560411    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 02 02:46:01.560536 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:01.560495    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert podName:e3d7f74d-20a3-484c-b9ca-4ab73b6a292c nodeName:}" failed. No retries permitted until 2026-06-02 02:46:17.560471595 +0000 UTC m=+66.176960679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-m6jzt" (UID: "e3d7f74d-20a3-484c-b9ca-4ab73b6a292c") : secret "networking-console-plugin-cert" not found
Jun 02 02:46:01.661503 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:01.661449    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:46:01.661660 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:01.661600    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 02:46:01.661660 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:01.661618    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d64b65bb-ndbb5: secret "image-registry-tls" not found
Jun 02 02:46:01.661733 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:01.661674    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls podName:6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:17.66165662 +0000 UTC m=+66.278145701 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls") pod "image-registry-55d64b65bb-ndbb5" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48") : secret "image-registry-tls" not found
Jun 02 02:46:01.762680 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:01.762647    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:46:01.762831 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:01.762692    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:46:01.762831 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:01.762796    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 02:46:01.762898 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:01.762853    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 02:46:01.762898 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:01.762859    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert podName:a457d44f-df4a-4ef5-9de4-632e3318feee nodeName:}" failed. No retries permitted until 2026-06-02 02:46:17.762843068 +0000 UTC m=+66.379332145 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert") pod "ingress-canary-nhglj" (UID: "a457d44f-df4a-4ef5-9de4-632e3318feee") : secret "canary-serving-cert" not found
Jun 02 02:46:01.762970 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:01.762909    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls podName:db50fd2b-9b3f-403f-a131-bf4a898ca513 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:17.762895475 +0000 UTC m=+66.379384543 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls") pod "dns-default-9x5hx" (UID: "db50fd2b-9b3f-403f-a131-bf4a898ca513") : secret "dns-default-metrics-tls" not found
Jun 02 02:46:12.178704 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:12.178679    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ws7xp"
Jun 02 02:46:17.587969 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.587916    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:46:17.588481 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.588071    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 02 02:46:17.588481 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.588146    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert podName:e3d7f74d-20a3-484c-b9ca-4ab73b6a292c nodeName:}" failed. No retries permitted until 2026-06-02 02:46:49.588130417 +0000 UTC m=+98.204619485 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-m6jzt" (UID: "e3d7f74d-20a3-484c-b9ca-4ab73b6a292c") : secret "networking-console-plugin-cert" not found
Jun 02 02:46:17.688760 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.688731    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:46:17.688923 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.688906    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 02:46:17.688989 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.688928    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d64b65bb-ndbb5: secret "image-registry-tls" not found
Jun 02 02:46:17.689040 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.689014    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls podName:6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:49.688990601 +0000 UTC m=+98.305479676 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls") pod "image-registry-55d64b65bb-ndbb5" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48") : secret "image-registry-tls" not found
Jun 02 02:46:17.789413 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.789378    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:46:17.789575 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.789441    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:46:17.789575 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.789492    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:46:17.789575 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.789533    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:46:17.789575 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.789556    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 02:46:17.789726 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.789626    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 02:46:17.789726 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.789628    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert podName:a457d44f-df4a-4ef5-9de4-632e3318feee nodeName:}" failed. No retries permitted until 2026-06-02 02:46:49.789608176 +0000 UTC m=+98.406097262 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert") pod "ingress-canary-nhglj" (UID: "a457d44f-df4a-4ef5-9de4-632e3318feee") : secret "canary-serving-cert" not found
Jun 02 02:46:17.789726 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.789677    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls podName:db50fd2b-9b3f-403f-a131-bf4a898ca513 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:49.789661121 +0000 UTC m=+98.406150195 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls") pod "dns-default-9x5hx" (UID: "db50fd2b-9b3f-403f-a131-bf4a898ca513") : secret "dns-default-metrics-tls" not found
Jun 02 02:46:17.792445 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.792427    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 02 02:46:17.792520 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.792496    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 02 02:46:17.799799 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.799784    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 02 02:46:17.799855 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:17.799827    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs podName:fcfae5f4-8103-45fa-beee-bdb243b5ee53 nodeName:}" failed. No retries permitted until 2026-06-02 02:47:21.799816783 +0000 UTC m=+130.416305853 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs") pod "network-metrics-daemon-vnj4x" (UID: "fcfae5f4-8103-45fa-beee-bdb243b5ee53") : secret "metrics-daemon-secret" not found
Jun 02 02:46:17.801889 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.801875    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 02 02:46:17.812934 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.812908    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxpdl\" (UniqueName: \"kubernetes.io/projected/861af5a6-480a-4494-89c0-0d4416a9fb9f-kube-api-access-bxpdl\") pod \"network-check-target-bcmlx\" (UID: \"861af5a6-480a-4494-89c0-0d4416a9fb9f\") " pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:46:17.940799 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.940727    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ldjq9\""
Jun 02 02:46:17.949064 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:17.949045    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:46:18.065930 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:18.065900    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bcmlx"]
Jun 02 02:46:18.069075 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:46:18.069043    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861af5a6_480a_4494_89c0_0d4416a9fb9f.slice/crio-ca9a0ed52c227debec45c8392b5c56804948a161eb08e5a1aa7f0d97f0b9285e WatchSource:0}: Error finding container ca9a0ed52c227debec45c8392b5c56804948a161eb08e5a1aa7f0d97f0b9285e: Status 404 returned error can't find the container with id ca9a0ed52c227debec45c8392b5c56804948a161eb08e5a1aa7f0d97f0b9285e
Jun 02 02:46:18.290065 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:18.289987    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bcmlx" event={"ID":"861af5a6-480a-4494-89c0-0d4416a9fb9f","Type":"ContainerStarted","Data":"ca9a0ed52c227debec45c8392b5c56804948a161eb08e5a1aa7f0d97f0b9285e"}
Jun 02 02:46:21.298060 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:21.298015    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bcmlx" event={"ID":"861af5a6-480a-4494-89c0-0d4416a9fb9f","Type":"ContainerStarted","Data":"f983b308ea0354e67bd79df713447cdc626ec726f3ebc76505ee70c7e92c2d58"}
Jun 02 02:46:21.298421 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:21.298147    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:46:21.315210 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:21.315122    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bcmlx" podStartSLOduration=66.335589092 podStartE2EDuration="1m9.31510998s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:46:18.071033281 +0000 UTC m=+66.687522365" lastFinishedPulling="2026-06-02 02:46:21.050554172 +0000 UTC m=+69.667043253" observedRunningTime="2026-06-02 02:46:21.313628598 +0000 UTC m=+69.930117689" watchObservedRunningTime="2026-06-02 02:46:21.31510998 +0000 UTC m=+69.931599101"
Jun 02 02:46:49.635828 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:49.635794    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:46:49.636321 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:49.635909    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 02 02:46:49.636321 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:49.636005    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert podName:e3d7f74d-20a3-484c-b9ca-4ab73b6a292c nodeName:}" failed. No retries permitted until 2026-06-02 02:47:53.635987065 +0000 UTC m=+162.252476133 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-m6jzt" (UID: "e3d7f74d-20a3-484c-b9ca-4ab73b6a292c") : secret "networking-console-plugin-cert" not found
Jun 02 02:46:49.736137 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:49.736101    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:46:49.736291 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:49.736268    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 02:46:49.736331 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:49.736293    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d64b65bb-ndbb5: secret "image-registry-tls" not found
Jun 02 02:46:49.736363 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:49.736358    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls podName:6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48 nodeName:}" failed. No retries permitted until 2026-06-02 02:47:53.736340991 +0000 UTC m=+162.352830061 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls") pod "image-registry-55d64b65bb-ndbb5" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48") : secret "image-registry-tls" not found
Jun 02 02:46:49.837033 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:49.837000    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:46:49.837143 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:49.837047    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:46:49.837183 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:49.837150    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 02:46:49.837215 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:49.837202    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 02:46:49.837246 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:49.837214    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert podName:a457d44f-df4a-4ef5-9de4-632e3318feee nodeName:}" failed. No retries permitted until 2026-06-02 02:47:53.83719793 +0000 UTC m=+162.453687014 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert") pod "ingress-canary-nhglj" (UID: "a457d44f-df4a-4ef5-9de4-632e3318feee") : secret "canary-serving-cert" not found
Jun 02 02:46:49.837285 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:46:49.837279    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls podName:db50fd2b-9b3f-403f-a131-bf4a898ca513 nodeName:}" failed. No retries permitted until 2026-06-02 02:47:53.837265546 +0000 UTC m=+162.453754615 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls") pod "dns-default-9x5hx" (UID: "db50fd2b-9b3f-403f-a131-bf4a898ca513") : secret "dns-default-metrics-tls" not found
Jun 02 02:46:52.303188 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:46:52.303154    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bcmlx"
Jun 02 02:47:21.882182 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:21.882141    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:47:21.882721 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:21.882301    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 02 02:47:21.882721 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:21.882384    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs podName:fcfae5f4-8103-45fa-beee-bdb243b5ee53 nodeName:}" failed. No retries permitted until 2026-06-02 02:49:23.882366186 +0000 UTC m=+252.498855259 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs") pod "network-metrics-daemon-vnj4x" (UID: "fcfae5f4-8103-45fa-beee-bdb243b5ee53") : secret "metrics-daemon-secret" not found
Jun 02 02:47:48.742643 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:48.742582    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt" podUID="e3d7f74d-20a3-484c-b9ca-4ab73b6a292c"
Jun 02 02:47:48.758747 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:48.758717    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5" podUID="6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48"
Jun 02 02:47:48.847566 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:48.847514    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9x5hx" podUID="db50fd2b-9b3f-403f-a131-bf4a898ca513"
Jun 02 02:47:48.869681 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:48.869648    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-nhglj" podUID="a457d44f-df4a-4ef5-9de4-632e3318feee"
Jun 02 02:47:49.502294 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:49.502213    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:47:49.502426 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:49.502316    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:47:49.502426 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:49.502347    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:47:49.502528 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:49.502512    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:47:50.044173 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:50.044129    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vnj4x" podUID="fcfae5f4-8103-45fa-beee-bdb243b5ee53"
Jun 02 02:47:52.365503 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:52.365474    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cn8tf_b170f48b-6b05-4361-90e3-4f5b81d273ea/dns-node-resolver/0.log"
Jun 02 02:47:53.371126 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:53.371100    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4x78s_23a084fd-4b0c-4c56-930d-b946a7e1bcbe/node-ca/0.log"
Jun 02 02:47:53.725782 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:53.725688    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:47:53.725942 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:53.725799    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jun 02 02:47:53.725942 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:53.725876    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert podName:e3d7f74d-20a3-484c-b9ca-4ab73b6a292c nodeName:}" failed. No retries permitted until 2026-06-02 02:49:55.725862069 +0000 UTC m=+284.342351143 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-m6jzt" (UID: "e3d7f74d-20a3-484c-b9ca-4ab73b6a292c") : secret "networking-console-plugin-cert" not found
Jun 02 02:47:53.826018 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:53.825984    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") pod \"image-registry-55d64b65bb-ndbb5\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") " pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:47:53.826150 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:53.826134    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 02 02:47:53.826150 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:53.826150    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d64b65bb-ndbb5: secret "image-registry-tls" not found
Jun 02 02:47:53.826220 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:53.826202    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls podName:6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48 nodeName:}" failed. No retries permitted until 2026-06-02 02:49:55.826185716 +0000 UTC m=+284.442674793 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls") pod "image-registry-55d64b65bb-ndbb5" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48") : secret "image-registry-tls" not found
Jun 02 02:47:53.926714 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:53.926676    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:47:53.926898 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:53.926771    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:47:53.926898 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:53.926839    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 02:47:53.926898 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:53.926855    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 02:47:53.927059 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:53.926905    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert podName:a457d44f-df4a-4ef5-9de4-632e3318feee nodeName:}" failed. No retries permitted until 2026-06-02 02:49:55.926891236 +0000 UTC m=+284.543380304 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert") pod "ingress-canary-nhglj" (UID: "a457d44f-df4a-4ef5-9de4-632e3318feee") : secret "canary-serving-cert" not found
Jun 02 02:47:53.927059 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:47:53.926916    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls podName:db50fd2b-9b3f-403f-a131-bf4a898ca513 nodeName:}" failed. No retries permitted until 2026-06-02 02:49:55.926910794 +0000 UTC m=+284.543399863 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls") pod "dns-default-9x5hx" (UID: "db50fd2b-9b3f-403f-a131-bf4a898ca513") : secret "dns-default-metrics-tls" not found
Jun 02 02:47:56.523210 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:56.523126    2554 generic.go:358] "Generic (PLEG): container finished" podID="317556c1-8dc7-48a5-aac4-31e57c41b6fe" containerID="7f71ea8773f589fb39f02cd7172744bb3512f9a65b44565f5d4029bf6de7fb00" exitCode=1
Jun 02 02:47:56.523679 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:56.523205    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk" event={"ID":"317556c1-8dc7-48a5-aac4-31e57c41b6fe","Type":"ContainerDied","Data":"7f71ea8773f589fb39f02cd7172744bb3512f9a65b44565f5d4029bf6de7fb00"}
Jun 02 02:47:56.523679 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:56.523615    2554 scope.go:117] "RemoveContainer" containerID="7f71ea8773f589fb39f02cd7172744bb3512f9a65b44565f5d4029bf6de7fb00"
Jun 02 02:47:56.524528 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:56.524509    2554 generic.go:358] "Generic (PLEG): container finished" podID="bb3655a7-5a4c-44fe-abd9-927b6b30adaf" containerID="37ee212c5324679670ef68abf327a5fc36febe0ae5cc437cf2d827dceb4157ad" exitCode=255
Jun 02 02:47:56.524645 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:56.524579    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf" event={"ID":"bb3655a7-5a4c-44fe-abd9-927b6b30adaf","Type":"ContainerDied","Data":"37ee212c5324679670ef68abf327a5fc36febe0ae5cc437cf2d827dceb4157ad"}
Jun 02 02:47:56.524910 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:56.524836    2554 scope.go:117] "RemoveContainer" containerID="37ee212c5324679670ef68abf327a5fc36febe0ae5cc437cf2d827dceb4157ad"
Jun 02 02:47:57.528434 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:57.528394    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk" event={"ID":"317556c1-8dc7-48a5-aac4-31e57c41b6fe","Type":"ContainerStarted","Data":"c737bb22025368c87799f196c57eb5328d313d9f1e13df08321c241e701dc2f5"}
Jun 02 02:47:57.528943 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:57.528759    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:47:57.529466 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:57.529429    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d65cd667c-4wsnk"
Jun 02 02:47:57.530055 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:47:57.530035    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74879db984-l9fkf" event={"ID":"bb3655a7-5a4c-44fe-abd9-927b6b30adaf","Type":"ContainerStarted","Data":"519a7b8389cf25614889f3966f456bd393af0c150d9104bc5e3a81edc30ddd6d"}
Jun 02 02:48:00.995700 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:00.995614    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:48:14.882663 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.882631    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-knmcm"]
Jun 02 02:48:14.884677 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.884660    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:14.887409 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.887389    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jun 02 02:48:14.887543 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.887409    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jun 02 02:48:14.888592 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.888571    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jun 02 02:48:14.888675 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.888642    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jun 02 02:48:14.888733 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.888707    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2sb7h\""
Jun 02 02:48:14.895882 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.895859    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-knmcm"]
Jun 02 02:48:14.990079 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.990046    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:14.990238 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.990182    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:14.990298 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.990245    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-crio-socket\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:14.990358 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.990323    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-data-volume\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:14.990410 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:14.990366    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpbj5\" (UniqueName: \"kubernetes.io/projected/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-kube-api-access-wpbj5\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.091096 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.091062    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.091281 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.091138    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-crio-socket\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.091281 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.091186    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-data-volume\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.091281 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.091218    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wpbj5\" (UniqueName: \"kubernetes.io/projected/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-kube-api-access-wpbj5\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.091281 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.091253    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.091499 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.091437    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-crio-socket\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.091691 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.091669    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-data-volume\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.091884 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.091864    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.093723 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.093699    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.099924 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.099903    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpbj5\" (UniqueName: \"kubernetes.io/projected/84bdde3a-4a50-45d9-b0ea-ea65680d0a06-kube-api-access-wpbj5\") pod \"insights-runtime-extractor-knmcm\" (UID: \"84bdde3a-4a50-45d9-b0ea-ea65680d0a06\") " pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.194116 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.194026    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-knmcm"
Jun 02 02:48:15.314811 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.314780    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-knmcm"]
Jun 02 02:48:15.318006 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:48:15.317977    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84bdde3a_4a50_45d9_b0ea_ea65680d0a06.slice/crio-f57ac255ddcff684badb91e23ff2cc34f26a6aa7aea462b41f545b960a375552 WatchSource:0}: Error finding container f57ac255ddcff684badb91e23ff2cc34f26a6aa7aea462b41f545b960a375552: Status 404 returned error can't find the container with id f57ac255ddcff684badb91e23ff2cc34f26a6aa7aea462b41f545b960a375552
Jun 02 02:48:15.570958 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.570924    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knmcm" event={"ID":"84bdde3a-4a50-45d9-b0ea-ea65680d0a06","Type":"ContainerStarted","Data":"5043ba2eae0512886f742ffe340d3076186aeb6ae971b42958b837c8d15692a4"}
Jun 02 02:48:15.570958 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:15.570962    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knmcm" event={"ID":"84bdde3a-4a50-45d9-b0ea-ea65680d0a06","Type":"ContainerStarted","Data":"f57ac255ddcff684badb91e23ff2cc34f26a6aa7aea462b41f545b960a375552"}
Jun 02 02:48:16.574646 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:16.574609    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knmcm" event={"ID":"84bdde3a-4a50-45d9-b0ea-ea65680d0a06","Type":"ContainerStarted","Data":"95fb7f60654ece2be2aa3b3afcfe187ab3c940e36a90eb4d73d39e4d6912672f"}
Jun 02 02:48:18.580392 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:18.580354    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knmcm" event={"ID":"84bdde3a-4a50-45d9-b0ea-ea65680d0a06","Type":"ContainerStarted","Data":"1a35cbfbfd2af8a648a62f7e032d63bc1d111bbec5f1a1102cf862769b938ead"}
Jun 02 02:48:18.600482 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:18.600417    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-knmcm" podStartSLOduration=2.211257431 podStartE2EDuration="4.600402591s" podCreationTimestamp="2026-06-02 02:48:14 +0000 UTC" firstStartedPulling="2026-06-02 02:48:15.36859739 +0000 UTC m=+183.985086459" lastFinishedPulling="2026-06-02 02:48:17.757742546 +0000 UTC m=+186.374231619" observedRunningTime="2026-06-02 02:48:18.598506598 +0000 UTC m=+187.214995689" watchObservedRunningTime="2026-06-02 02:48:18.600402591 +0000 UTC m=+187.216891682"
Jun 02 02:48:30.421954 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.421911    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8lrv9"]
Jun 02 02:48:30.425087 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.425063    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.428297 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.428276    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jun 02 02:48:30.428408 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.428315    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jun 02 02:48:30.428559 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.428544    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jun 02 02:48:30.429726 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.429704    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jun 02 02:48:30.429829 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.429784    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jun 02 02:48:30.429892 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.429826    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mcxww\""
Jun 02 02:48:30.429892 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.429842    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jun 02 02:48:30.513724 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.513694    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm26g\" (UniqueName: \"kubernetes.io/projected/d4af44da-ca2e-4e34-8828-0a2791b8d274-kube-api-access-pm26g\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.513724 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.513731    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.513984 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.513766    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4af44da-ca2e-4e34-8828-0a2791b8d274-sys\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.513984 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.513864    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-tls\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.513984 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.513899    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4af44da-ca2e-4e34-8828-0a2791b8d274-metrics-client-ca\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.513984 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.513929    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-wtmp\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.514144 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.513985    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d4af44da-ca2e-4e34-8828-0a2791b8d274-root\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.514144 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.514052    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-textfile\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.514144 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.514082    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-accelerators-collector-config\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615183 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615156    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4af44da-ca2e-4e34-8828-0a2791b8d274-sys\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615341 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615209    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-tls\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615341 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615227    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4af44da-ca2e-4e34-8828-0a2791b8d274-metrics-client-ca\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615341 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615246    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-wtmp\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615341 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615258    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4af44da-ca2e-4e34-8828-0a2791b8d274-sys\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615341 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615288    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d4af44da-ca2e-4e34-8828-0a2791b8d274-root\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615341 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615328    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-textfile\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615648 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615355    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-accelerators-collector-config\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615648 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615368    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-wtmp\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615648 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:48:30.615377    2554 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jun 02 02:48:30.615648 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615412    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pm26g\" (UniqueName: \"kubernetes.io/projected/d4af44da-ca2e-4e34-8828-0a2791b8d274-kube-api-access-pm26g\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615648 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615436    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d4af44da-ca2e-4e34-8828-0a2791b8d274-root\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615648 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:48:30.615446    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-tls podName:d4af44da-ca2e-4e34-8828-0a2791b8d274 nodeName:}" failed. No retries permitted until 2026-06-02 02:48:31.115427282 +0000 UTC m=+199.731916367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-tls") pod "node-exporter-8lrv9" (UID: "d4af44da-ca2e-4e34-8828-0a2791b8d274") : secret "node-exporter-tls" not found
Jun 02 02:48:30.615648 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615510    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615887 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615703    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-textfile\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615926 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615900    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4af44da-ca2e-4e34-8828-0a2791b8d274-metrics-client-ca\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.615959 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.615902    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-accelerators-collector-config\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.617669 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.617651    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:30.632517 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:30.632499    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm26g\" (UniqueName: \"kubernetes.io/projected/d4af44da-ca2e-4e34-8828-0a2791b8d274-kube-api-access-pm26g\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:31.119930 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:31.119889    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-tls\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:31.120118 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:48:31.120092    2554 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jun 02 02:48:31.120216 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:48:31.120201    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-tls podName:d4af44da-ca2e-4e34-8828-0a2791b8d274 nodeName:}" failed. No retries permitted until 2026-06-02 02:48:32.120178903 +0000 UTC m=+200.736667992 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-tls") pod "node-exporter-8lrv9" (UID: "d4af44da-ca2e-4e34-8828-0a2791b8d274") : secret "node-exporter-tls" not found
Jun 02 02:48:32.127088 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:32.127053    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-tls\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:32.129423 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:32.129399    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d4af44da-ca2e-4e34-8828-0a2791b8d274-node-exporter-tls\") pod \"node-exporter-8lrv9\" (UID: \"d4af44da-ca2e-4e34-8828-0a2791b8d274\") " pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:32.234295 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:32.234260    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8lrv9"
Jun 02 02:48:32.242233 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:48:32.242206    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4af44da_ca2e_4e34_8828_0a2791b8d274.slice/crio-c7064dbc57a2a5b3e4b1ff9ea2d3b01a5d98683b2a117fac619df87978041128 WatchSource:0}: Error finding container c7064dbc57a2a5b3e4b1ff9ea2d3b01a5d98683b2a117fac619df87978041128: Status 404 returned error can't find the container with id c7064dbc57a2a5b3e4b1ff9ea2d3b01a5d98683b2a117fac619df87978041128
Jun 02 02:48:32.614487 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:32.614436    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lrv9" event={"ID":"d4af44da-ca2e-4e34-8828-0a2791b8d274","Type":"ContainerStarted","Data":"c7064dbc57a2a5b3e4b1ff9ea2d3b01a5d98683b2a117fac619df87978041128"}
Jun 02 02:48:33.618920 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:33.618884    2554 generic.go:358] "Generic (PLEG): container finished" podID="d4af44da-ca2e-4e34-8828-0a2791b8d274" containerID="ca71e8412f4ffdf7a88671b3451314d98d395e5c4dec4fbecca070b87d41c5d5" exitCode=0
Jun 02 02:48:33.619340 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:33.618932    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lrv9" event={"ID":"d4af44da-ca2e-4e34-8828-0a2791b8d274","Type":"ContainerDied","Data":"ca71e8412f4ffdf7a88671b3451314d98d395e5c4dec4fbecca070b87d41c5d5"}
Jun 02 02:48:34.622838 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:34.622798    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lrv9" event={"ID":"d4af44da-ca2e-4e34-8828-0a2791b8d274","Type":"ContainerStarted","Data":"3dff4be78c31659d55479095b93b4e5732f96d3dd7af00ca5463736c46cacc86"}
Jun 02 02:48:34.622838 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:34.622833    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lrv9" event={"ID":"d4af44da-ca2e-4e34-8828-0a2791b8d274","Type":"ContainerStarted","Data":"902595416d3486a340d739e572ca5987a05b6c7b837356d432f63adb29b69f2b"}
Jun 02 02:48:34.644733 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:34.644689    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8lrv9" podStartSLOduration=3.665542416 podStartE2EDuration="4.644677995s" podCreationTimestamp="2026-06-02 02:48:30 +0000 UTC" firstStartedPulling="2026-06-02 02:48:32.24393164 +0000 UTC m=+200.860420709" lastFinishedPulling="2026-06-02 02:48:33.223067206 +0000 UTC m=+201.839556288" observedRunningTime="2026-06-02 02:48:34.643861137 +0000 UTC m=+203.260350229" watchObservedRunningTime="2026-06-02 02:48:34.644677995 +0000 UTC m=+203.261167112"
Jun 02 02:48:37.014590 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.014556    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55d64b65bb-ndbb5"]
Jun 02 02:48:37.014953 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:48:37.014753    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5" podUID="6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48"
Jun 02 02:48:37.630156 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.630120    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:48:37.635358 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.635336    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:48:37.773333 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.773298    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-ca-trust-extracted\") pod \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") "
Jun 02 02:48:37.773516 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.773348    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-bound-sa-token\") pod \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") "
Jun 02 02:48:37.773516 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.773374    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-installation-pull-secrets\") pod \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") "
Jun 02 02:48:37.773516 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.773413    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zthf\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-kube-api-access-6zthf\") pod \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") "
Jun 02 02:48:37.773516 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.773444    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-image-registry-private-configuration\") pod \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") "
Jun 02 02:48:37.773516 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.773512    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-certificates\") pod \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") "
Jun 02 02:48:37.773764 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.773532    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-trusted-ca\") pod \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\" (UID: \"6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48\") "
Jun 02 02:48:37.773764 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.773627    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 02:48:37.773936 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.773900    2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-ca-trust-extracted\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:48:37.774194 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.774165    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 02:48:37.774298 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.774237    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 02:48:37.775741 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.775719    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 02:48:37.775817 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.775779    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-kube-api-access-6zthf" (OuterVolumeSpecName: "kube-api-access-6zthf") pod "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48"). InnerVolumeSpecName "kube-api-access-6zthf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 02:48:37.776100 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.776082    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 02:48:37.776170 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.776117    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48" (UID: "6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 02:48:37.875018 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.874984    2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-certificates\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:48:37.875018 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.875011    2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-trusted-ca\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:48:37.875018 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.875023    2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-bound-sa-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:48:37.875248 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.875033    2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-installation-pull-secrets\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:48:37.875248 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.875042    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zthf\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-kube-api-access-6zthf\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:48:37.875248 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:37.875051    2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-image-registry-private-configuration\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:48:38.632662 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:38.632631    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d64b65bb-ndbb5"
Jun 02 02:48:38.682684 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:38.682655    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55d64b65bb-ndbb5"]
Jun 02 02:48:38.687670 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:38.687643    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55d64b65bb-ndbb5"]
Jun 02 02:48:38.781675 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:38.781644    2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48-registry-tls\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:48:39.999163 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:39.999126    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48" path="/var/lib/kubelet/pods/6a3dfcc3-c86a-465d-8ca6-1a44e09f6d48/volumes"
Jun 02 02:48:46.094664 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:46.094624    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" podUID="ec92eb63-9f2a-468a-aafc-e57f6fdec868" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jun 02 02:48:56.094267 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:48:56.094224    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" podUID="ec92eb63-9f2a-468a-aafc-e57f6fdec868" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jun 02 02:49:06.094521 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:06.094479    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" podUID="ec92eb63-9f2a-468a-aafc-e57f6fdec868" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jun 02 02:49:06.094987 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:06.094553    2554 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j"
Jun 02 02:49:06.095030 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:06.094983    2554 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"aff3809d85d078c594733ace6c02674a6bff580401d3ca280454c6c0c5b8464e"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jun 02 02:49:06.095065 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:06.095029    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" podUID="ec92eb63-9f2a-468a-aafc-e57f6fdec868" containerName="service-proxy" containerID="cri-o://aff3809d85d078c594733ace6c02674a6bff580401d3ca280454c6c0c5b8464e" gracePeriod=30
Jun 02 02:49:06.706512 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:06.706469    2554 generic.go:358] "Generic (PLEG): container finished" podID="ec92eb63-9f2a-468a-aafc-e57f6fdec868" containerID="aff3809d85d078c594733ace6c02674a6bff580401d3ca280454c6c0c5b8464e" exitCode=2
Jun 02 02:49:06.706512 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:06.706491    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" event={"ID":"ec92eb63-9f2a-468a-aafc-e57f6fdec868","Type":"ContainerDied","Data":"aff3809d85d078c594733ace6c02674a6bff580401d3ca280454c6c0c5b8464e"}
Jun 02 02:49:06.706736 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:06.706529    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6865c5b45c-gmb6j" event={"ID":"ec92eb63-9f2a-468a-aafc-e57f6fdec868","Type":"ContainerStarted","Data":"ae12aee26fb2bde6afaa5aa5760c0995e88d29cfb81772cbb20d4dc4ca8917ea"}
Jun 02 02:49:23.924331 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:23.924268    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:49:23.926676 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:23.926654    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcfae5f4-8103-45fa-beee-bdb243b5ee53-metrics-certs\") pod \"network-metrics-daemon-vnj4x\" (UID: \"fcfae5f4-8103-45fa-beee-bdb243b5ee53\") " pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:49:24.099625 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:24.099590    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgk2p\""
Jun 02 02:49:24.106963 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:24.106945    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vnj4x"
Jun 02 02:49:24.230475 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:24.230435    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vnj4x"]
Jun 02 02:49:24.234367 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:49:24.234343    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfae5f4_8103_45fa_beee_bdb243b5ee53.slice/crio-1bab03cb4b4cc6b2bb3e8e2caef0bae07b6374bd77db722cdbe96c218605c874 WatchSource:0}: Error finding container 1bab03cb4b4cc6b2bb3e8e2caef0bae07b6374bd77db722cdbe96c218605c874: Status 404 returned error can't find the container with id 1bab03cb4b4cc6b2bb3e8e2caef0bae07b6374bd77db722cdbe96c218605c874
Jun 02 02:49:24.751437 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:24.751401    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vnj4x" event={"ID":"fcfae5f4-8103-45fa-beee-bdb243b5ee53","Type":"ContainerStarted","Data":"1bab03cb4b4cc6b2bb3e8e2caef0bae07b6374bd77db722cdbe96c218605c874"}
Jun 02 02:49:25.756344 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:25.756259    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vnj4x" event={"ID":"fcfae5f4-8103-45fa-beee-bdb243b5ee53","Type":"ContainerStarted","Data":"11ac78929a35a96dd66ad6fb921b2cb5231652ba6d9d243f0d9691b6540ca0e5"}
Jun 02 02:49:25.756344 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:25.756294    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vnj4x" event={"ID":"fcfae5f4-8103-45fa-beee-bdb243b5ee53","Type":"ContainerStarted","Data":"0a25c3c27f2917bc60c89fb2cbdae13f727e2a51bb7c86bcbfbd9b12913d03a9"}
Jun 02 02:49:25.772020 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:25.771978    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vnj4x" podStartSLOduration=252.577959151 podStartE2EDuration="4m13.77196432s" podCreationTimestamp="2026-06-02 02:45:12 +0000 UTC" firstStartedPulling="2026-06-02 02:49:24.236191944 +0000 UTC m=+252.852681014" lastFinishedPulling="2026-06-02 02:49:25.430197111 +0000 UTC m=+254.046686183" observedRunningTime="2026-06-02 02:49:25.771354124 +0000 UTC m=+254.387843215" watchObservedRunningTime="2026-06-02 02:49:25.77196432 +0000 UTC m=+254.388453410"
Jun 02 02:49:52.502755 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:49:52.502709    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-nhglj" podUID="a457d44f-df4a-4ef5-9de4-632e3318feee"
Jun 02 02:49:52.503157 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:49:52.502763    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt" podUID="e3d7f74d-20a3-484c-b9ca-4ab73b6a292c"
Jun 02 02:49:52.503157 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:49:52.502923    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9x5hx" podUID="db50fd2b-9b3f-403f-a131-bf4a898ca513"
Jun 02 02:49:52.827419 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:52.827380    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:49:52.827620 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:52.827380    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:49:52.827620 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:52.827380    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:49:55.773328 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:55.773293    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:49:55.775741 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:55.775700    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3d7f74d-20a3-484c-b9ca-4ab73b6a292c-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-m6jzt\" (UID: \"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:49:55.831670 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:55.831641    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vmqqx\""
Jun 02 02:49:55.839006 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:55.838980    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"
Jun 02 02:49:55.958496 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:55.958472    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt"]
Jun 02 02:49:55.961360 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:49:55.961327    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d7f74d_20a3_484c_b9ca_4ab73b6a292c.slice/crio-803aa1e2fc39a3a1b7a902809595d046d2161e226183ed1f5aa9a7a421defe82 WatchSource:0}: Error finding container 803aa1e2fc39a3a1b7a902809595d046d2161e226183ed1f5aa9a7a421defe82: Status 404 returned error can't find the container with id 803aa1e2fc39a3a1b7a902809595d046d2161e226183ed1f5aa9a7a421defe82
Jun 02 02:49:55.974156 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:55.974133    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:49:55.974251 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:55.974224    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:49:55.976306 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:55.976282    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db50fd2b-9b3f-403f-a131-bf4a898ca513-metrics-tls\") pod \"dns-default-9x5hx\" (UID: \"db50fd2b-9b3f-403f-a131-bf4a898ca513\") " pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:49:55.976586 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:55.976571    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a457d44f-df4a-4ef5-9de4-632e3318feee-cert\") pod \"ingress-canary-nhglj\" (UID: \"a457d44f-df4a-4ef5-9de4-632e3318feee\") " pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:49:56.130882 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:56.130853    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5469w\""
Jun 02 02:49:56.130882 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:56.130853    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kcfjp\""
Jun 02 02:49:56.138143 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:56.138115    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nhglj"
Jun 02 02:49:56.138143 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:56.138143    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:49:56.265684 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:56.265645    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9x5hx"]
Jun 02 02:49:56.268416 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:49:56.268388    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb50fd2b_9b3f_403f_a131_bf4a898ca513.slice/crio-9c2518827dc4cb50f4d06109a95cebc985df27276d43769331d2ccd97a0e181d WatchSource:0}: Error finding container 9c2518827dc4cb50f4d06109a95cebc985df27276d43769331d2ccd97a0e181d: Status 404 returned error can't find the container with id 9c2518827dc4cb50f4d06109a95cebc985df27276d43769331d2ccd97a0e181d
Jun 02 02:49:56.283167 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:56.283146    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nhglj"]
Jun 02 02:49:56.286025 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:49:56.286002    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda457d44f_df4a_4ef5_9de4_632e3318feee.slice/crio-fa9701d60a0d9555afff81a0ff9293863b2144fd525e3129c7438a490e4e2e98 WatchSource:0}: Error finding container fa9701d60a0d9555afff81a0ff9293863b2144fd525e3129c7438a490e4e2e98: Status 404 returned error can't find the container with id fa9701d60a0d9555afff81a0ff9293863b2144fd525e3129c7438a490e4e2e98
Jun 02 02:49:56.838523 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:56.838491    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nhglj" event={"ID":"a457d44f-df4a-4ef5-9de4-632e3318feee","Type":"ContainerStarted","Data":"fa9701d60a0d9555afff81a0ff9293863b2144fd525e3129c7438a490e4e2e98"}
Jun 02 02:49:56.839902 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:56.839868    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9x5hx" event={"ID":"db50fd2b-9b3f-403f-a131-bf4a898ca513","Type":"ContainerStarted","Data":"9c2518827dc4cb50f4d06109a95cebc985df27276d43769331d2ccd97a0e181d"}
Jun 02 02:49:56.841002 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:56.840973    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt" event={"ID":"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c","Type":"ContainerStarted","Data":"803aa1e2fc39a3a1b7a902809595d046d2161e226183ed1f5aa9a7a421defe82"}
Jun 02 02:49:57.846640 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:57.846601    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt" event={"ID":"e3d7f74d-20a3-484c-b9ca-4ab73b6a292c","Type":"ContainerStarted","Data":"63f77187658d7bb0e3d4a9304c428bcf30e1f3db5723b472691977a6a9541044"}
Jun 02 02:49:57.863832 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:57.863772    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-m6jzt" podStartSLOduration=267.583504099 podStartE2EDuration="4m28.863752556s" podCreationTimestamp="2026-06-02 02:45:29 +0000 UTC" firstStartedPulling="2026-06-02 02:49:55.963682976 +0000 UTC m=+284.580172045" lastFinishedPulling="2026-06-02 02:49:57.243931434 +0000 UTC m=+285.860420502" observedRunningTime="2026-06-02 02:49:57.862215386 +0000 UTC m=+286.478704477" watchObservedRunningTime="2026-06-02 02:49:57.863752556 +0000 UTC m=+286.480241692"
Jun 02 02:49:58.850762 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:58.850722    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nhglj" event={"ID":"a457d44f-df4a-4ef5-9de4-632e3318feee","Type":"ContainerStarted","Data":"688795ff347426b6c90ebd9ec482a831ccac67e370e6d46abf4bf3e0c72ec39a"}
Jun 02 02:49:58.852235 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:58.852208    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9x5hx" event={"ID":"db50fd2b-9b3f-403f-a131-bf4a898ca513","Type":"ContainerStarted","Data":"e09daad72b91251215e8e06e60f574fe88b0f4e874559f6e7bcfd89d797356c3"}
Jun 02 02:49:58.852336 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:58.852242    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9x5hx" event={"ID":"db50fd2b-9b3f-403f-a131-bf4a898ca513","Type":"ContainerStarted","Data":"47e11076b12f14ef6c484abc83c58969267e90ea06ddb4681cace3b1fd52dc85"}
Jun 02 02:49:58.868283 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:58.868236    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nhglj" podStartSLOduration=251.857913793 podStartE2EDuration="4m13.868221149s" podCreationTimestamp="2026-06-02 02:45:45 +0000 UTC" firstStartedPulling="2026-06-02 02:49:56.287750794 +0000 UTC m=+284.904239863" lastFinishedPulling="2026-06-02 02:49:58.298058132 +0000 UTC m=+286.914547219" observedRunningTime="2026-06-02 02:49:58.867352645 +0000 UTC m=+287.483841746" watchObservedRunningTime="2026-06-02 02:49:58.868221149 +0000 UTC m=+287.484710236"
Jun 02 02:49:58.884798 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:58.884754    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-9x5hx" podStartSLOduration=251.855409619 podStartE2EDuration="4m13.884740946s" podCreationTimestamp="2026-06-02 02:45:45 +0000 UTC" firstStartedPulling="2026-06-02 02:49:56.270326915 +0000 UTC m=+284.886815983" lastFinishedPulling="2026-06-02 02:49:58.299658236 +0000 UTC m=+286.916147310" observedRunningTime="2026-06-02 02:49:58.883550119 +0000 UTC m=+287.500039211" watchObservedRunningTime="2026-06-02 02:49:58.884740946 +0000 UTC m=+287.501230037"
Jun 02 02:49:59.855509 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:49:59.855478    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:50:09.860807 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:50:09.860777    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9x5hx"
Jun 02 02:50:11.870996 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:50:11.870966    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 02:50:11.871390 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:50:11.870966    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 02:50:11.875471 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:50:11.875434    2554 kubelet.go:1628] "Image garbage collection succeeded"
Jun 02 02:51:49.498013 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.497917    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-zfdkw"]
Jun 02 02:51:49.500817 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.500794    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw"
Jun 02 02:51:49.503511 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.503493    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jun 02 02:51:49.503612 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.503497    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9rhxl\""
Jun 02 02:51:49.504605 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.504590    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jun 02 02:51:49.510820 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.510794    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-zfdkw"]
Jun 02 02:51:49.554968 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.554936    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf9dac02-775d-48ab-9f20-eadf33b9a4ac-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-zfdkw\" (UID: \"bf9dac02-775d-48ab-9f20-eadf33b9a4ac\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw"
Jun 02 02:51:49.555089 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.554976    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grtn9\" (UniqueName: \"kubernetes.io/projected/bf9dac02-775d-48ab-9f20-eadf33b9a4ac-kube-api-access-grtn9\") pod \"cert-manager-cainjector-68b757865b-zfdkw\" (UID: \"bf9dac02-775d-48ab-9f20-eadf33b9a4ac\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw"
Jun 02 02:51:49.655569 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.655530    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf9dac02-775d-48ab-9f20-eadf33b9a4ac-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-zfdkw\" (UID: \"bf9dac02-775d-48ab-9f20-eadf33b9a4ac\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw"
Jun 02 02:51:49.655569 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.655576    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-grtn9\" (UniqueName: \"kubernetes.io/projected/bf9dac02-775d-48ab-9f20-eadf33b9a4ac-kube-api-access-grtn9\") pod \"cert-manager-cainjector-68b757865b-zfdkw\" (UID: \"bf9dac02-775d-48ab-9f20-eadf33b9a4ac\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw"
Jun 02 02:51:49.664702 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.664674    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf9dac02-775d-48ab-9f20-eadf33b9a4ac-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-zfdkw\" (UID: \"bf9dac02-775d-48ab-9f20-eadf33b9a4ac\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw"
Jun 02 02:51:49.665003 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.664981    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grtn9\" (UniqueName: \"kubernetes.io/projected/bf9dac02-775d-48ab-9f20-eadf33b9a4ac-kube-api-access-grtn9\") pod \"cert-manager-cainjector-68b757865b-zfdkw\" (UID: \"bf9dac02-775d-48ab-9f20-eadf33b9a4ac\") " pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw"
Jun 02 02:51:49.809674 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.809635    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw"
Jun 02 02:51:49.930547 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.930519    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-zfdkw"]
Jun 02 02:51:49.933582 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:51:49.933556    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf9dac02_775d_48ab_9f20_eadf33b9a4ac.slice/crio-c8bd801dbf7aa9d1fc8064fcd5c78b5a4eb929c4da8754b330391da643f8e725 WatchSource:0}: Error finding container c8bd801dbf7aa9d1fc8064fcd5c78b5a4eb929c4da8754b330391da643f8e725: Status 404 returned error can't find the container with id c8bd801dbf7aa9d1fc8064fcd5c78b5a4eb929c4da8754b330391da643f8e725
Jun 02 02:51:49.935530 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:49.935511    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 02:51:50.123437 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:50.123352    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw" event={"ID":"bf9dac02-775d-48ab-9f20-eadf33b9a4ac","Type":"ContainerStarted","Data":"c8bd801dbf7aa9d1fc8064fcd5c78b5a4eb929c4da8754b330391da643f8e725"}
Jun 02 02:51:53.134181 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:53.134139    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw" event={"ID":"bf9dac02-775d-48ab-9f20-eadf33b9a4ac","Type":"ContainerStarted","Data":"129cf5a8938569f5678fbbb988d012bdf70e5eed2400ac48553d477c5b899d85"}
Jun 02 02:51:53.150436 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:51:53.150385    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-zfdkw" podStartSLOduration=1.367797908 podStartE2EDuration="4.150371673s" podCreationTimestamp="2026-06-02 02:51:49 +0000 UTC" firstStartedPulling="2026-06-02 02:51:49.935641098 +0000 UTC m=+398.552130167" lastFinishedPulling="2026-06-02 02:51:52.718214863 +0000 UTC m=+401.334703932" observedRunningTime="2026-06-02 02:51:53.149028395 +0000 UTC m=+401.765517485" watchObservedRunningTime="2026-06-02 02:51:53.150371673 +0000 UTC m=+401.766860765"
Jun 02 02:52:17.441891 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.441852    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"]
Jun 02 02:52:17.445862 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.445846    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.448378 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.448359    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jun 02 02:52:17.449028 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.449011    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jun 02 02:52:17.450011 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.449980    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4m85r\""
Jun 02 02:52:17.450114 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.450049    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jun 02 02:52:17.450174 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.450141    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jun 02 02:52:17.465948 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.465929    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"]
Jun 02 02:52:17.542650 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.542619    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/521bc591-2a8d-4436-92f3-ec2923cdad98-manifests\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.542802 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.542673    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/521bc591-2a8d-4436-92f3-ec2923cdad98-tmp\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.542802 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.542731    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkfmr\" (UniqueName: \"kubernetes.io/projected/521bc591-2a8d-4436-92f3-ec2923cdad98-kube-api-access-wkfmr\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.542802 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.542788    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/521bc591-2a8d-4436-92f3-ec2923cdad98-webhook-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.542904 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.542823    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/521bc591-2a8d-4436-92f3-ec2923cdad98-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.643466 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.643415    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wkfmr\" (UniqueName: \"kubernetes.io/projected/521bc591-2a8d-4436-92f3-ec2923cdad98-kube-api-access-wkfmr\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.643668 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.643483    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/521bc591-2a8d-4436-92f3-ec2923cdad98-webhook-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.643668 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.643514    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/521bc591-2a8d-4436-92f3-ec2923cdad98-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.643668 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.643540    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/521bc591-2a8d-4436-92f3-ec2923cdad98-manifests\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.643668 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.643582    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/521bc591-2a8d-4436-92f3-ec2923cdad98-tmp\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.644025 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.643998    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/521bc591-2a8d-4436-92f3-ec2923cdad98-manifests\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.645754 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.645731    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/521bc591-2a8d-4436-92f3-ec2923cdad98-tmp\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.646013 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.645995    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/521bc591-2a8d-4436-92f3-ec2923cdad98-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.646086 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.646069    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/521bc591-2a8d-4436-92f3-ec2923cdad98-webhook-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.658002 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.657972    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkfmr\" (UniqueName: \"kubernetes.io/projected/521bc591-2a8d-4436-92f3-ec2923cdad98-kube-api-access-wkfmr\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-vnlqd\" (UID: \"521bc591-2a8d-4436-92f3-ec2923cdad98\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.755570 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.755492    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:17.880770 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:17.880736    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"]
Jun 02 02:52:17.883684 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:52:17.883659    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521bc591_2a8d_4436_92f3_ec2923cdad98.slice/crio-79e4ba40ee1e37abb97ad6a532e87e83175c121aac052d1f7ec288cdd4b2ea33 WatchSource:0}: Error finding container 79e4ba40ee1e37abb97ad6a532e87e83175c121aac052d1f7ec288cdd4b2ea33: Status 404 returned error can't find the container with id 79e4ba40ee1e37abb97ad6a532e87e83175c121aac052d1f7ec288cdd4b2ea33
Jun 02 02:52:18.197625 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:18.197586    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd" event={"ID":"521bc591-2a8d-4436-92f3-ec2923cdad98","Type":"ContainerStarted","Data":"79e4ba40ee1e37abb97ad6a532e87e83175c121aac052d1f7ec288cdd4b2ea33"}
Jun 02 02:52:21.208318 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:21.208283    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd" event={"ID":"521bc591-2a8d-4436-92f3-ec2923cdad98","Type":"ContainerStarted","Data":"bda7e1bc568c01e678981a4fb7f5eb5f80490fc8e16ee3dfa8aa67e9ff1ef9b7"}
Jun 02 02:52:22.211955 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:22.211918    2554 generic.go:358] "Generic (PLEG): container finished" podID="521bc591-2a8d-4436-92f3-ec2923cdad98" containerID="bda7e1bc568c01e678981a4fb7f5eb5f80490fc8e16ee3dfa8aa67e9ff1ef9b7" exitCode=0
Jun 02 02:52:22.212337 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:22.211961    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd" event={"ID":"521bc591-2a8d-4436-92f3-ec2923cdad98","Type":"ContainerDied","Data":"bda7e1bc568c01e678981a4fb7f5eb5f80490fc8e16ee3dfa8aa67e9ff1ef9b7"}
Jun 02 02:52:23.216062 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:23.216025    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd" event={"ID":"521bc591-2a8d-4436-92f3-ec2923cdad98","Type":"ContainerStarted","Data":"93eb014a856e541f3732f46777eda7f4e109e49d621a7946f560bf6ea4574090"}
Jun 02 02:52:23.216517 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:23.216178    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:23.237658 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:23.237611    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd" podStartSLOduration=1.589012315 podStartE2EDuration="6.237597904s" podCreationTimestamp="2026-06-02 02:52:17 +0000 UTC" firstStartedPulling="2026-06-02 02:52:17.885336164 +0000 UTC m=+426.501825234" lastFinishedPulling="2026-06-02 02:52:22.533921743 +0000 UTC m=+431.150410823" observedRunningTime="2026-06-02 02:52:23.236049293 +0000 UTC m=+431.852538394" watchObservedRunningTime="2026-06-02 02:52:23.237597904 +0000 UTC m=+431.854086994"
Jun 02 02:52:34.220545 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:34.220448    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-vnlqd"
Jun 02 02:52:36.211639 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.211600    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"]
Jun 02 02:52:36.214692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.214669    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.217398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.217376    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jun 02 02:52:36.217524 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.217372    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jun 02 02:52:36.217680 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.217663    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jun 02 02:52:36.218770 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.218751    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6g6sv\""
Jun 02 02:52:36.218876 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.218803    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jun 02 02:52:36.224204 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.224180    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"]
Jun 02 02:52:36.279865 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.279824    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99546707-eb92-4ba6-a09b-e33b2186434c-tls-certs\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.280034 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.279896    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch2xn\" (UniqueName: \"kubernetes.io/projected/99546707-eb92-4ba6-a09b-e33b2186434c-kube-api-access-ch2xn\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.280034 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.279920    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99546707-eb92-4ba6-a09b-e33b2186434c-tmp\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.380850 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.380807    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99546707-eb92-4ba6-a09b-e33b2186434c-tls-certs\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.381013 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.380906    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ch2xn\" (UniqueName: \"kubernetes.io/projected/99546707-eb92-4ba6-a09b-e33b2186434c-kube-api-access-ch2xn\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.381013 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.380939    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99546707-eb92-4ba6-a09b-e33b2186434c-tmp\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.381013 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:52:36.380963    2554 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jun 02 02:52:36.381110 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:52:36.381046    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99546707-eb92-4ba6-a09b-e33b2186434c-tls-certs podName:99546707-eb92-4ba6-a09b-e33b2186434c nodeName:}" failed. No retries permitted until 2026-06-02 02:52:36.881025798 +0000 UTC m=+445.497514873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/99546707-eb92-4ba6-a09b-e33b2186434c-tls-certs") pod "kube-auth-proxy-85d86d5b57-2264c" (UID: "99546707-eb92-4ba6-a09b-e33b2186434c") : secret "kube-auth-proxy-tls" not found
Jun 02 02:52:36.383093 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.383075    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99546707-eb92-4ba6-a09b-e33b2186434c-tmp\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.398568 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.398546    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch2xn\" (UniqueName: \"kubernetes.io/projected/99546707-eb92-4ba6-a09b-e33b2186434c-kube-api-access-ch2xn\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.449246 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.449213    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-2qq4b"]
Jun 02 02:52:36.452248 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.452232    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:36.455261 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.455239    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jun 02 02:52:36.455389 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.455321    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tcxl7\""
Jun 02 02:52:36.463479 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.463400    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-2qq4b"]
Jun 02 02:52:36.582726 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.582691    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzxvq\" (UniqueName: \"kubernetes.io/projected/0e25396f-883d-4114-8451-8359c3ecf4f0-kube-api-access-jzxvq\") pod \"odh-model-controller-76588b95-2qq4b\" (UID: \"0e25396f-883d-4114-8451-8359c3ecf4f0\") " pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:36.582726 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.582732    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e25396f-883d-4114-8451-8359c3ecf4f0-cert\") pod \"odh-model-controller-76588b95-2qq4b\" (UID: \"0e25396f-883d-4114-8451-8359c3ecf4f0\") " pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:36.683863 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.683833    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzxvq\" (UniqueName: \"kubernetes.io/projected/0e25396f-883d-4114-8451-8359c3ecf4f0-kube-api-access-jzxvq\") pod \"odh-model-controller-76588b95-2qq4b\" (UID: \"0e25396f-883d-4114-8451-8359c3ecf4f0\") " pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:36.684016 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.683880    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e25396f-883d-4114-8451-8359c3ecf4f0-cert\") pod \"odh-model-controller-76588b95-2qq4b\" (UID: \"0e25396f-883d-4114-8451-8359c3ecf4f0\") " pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:36.684016 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:52:36.683989    2554 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jun 02 02:52:36.684112 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:52:36.684045    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e25396f-883d-4114-8451-8359c3ecf4f0-cert podName:0e25396f-883d-4114-8451-8359c3ecf4f0 nodeName:}" failed. No retries permitted until 2026-06-02 02:52:37.184028666 +0000 UTC m=+445.800517756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0e25396f-883d-4114-8451-8359c3ecf4f0-cert") pod "odh-model-controller-76588b95-2qq4b" (UID: "0e25396f-883d-4114-8451-8359c3ecf4f0") : secret "odh-model-controller-webhook-cert" not found
Jun 02 02:52:36.693253 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.693228    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzxvq\" (UniqueName: \"kubernetes.io/projected/0e25396f-883d-4114-8451-8359c3ecf4f0-kube-api-access-jzxvq\") pod \"odh-model-controller-76588b95-2qq4b\" (UID: \"0e25396f-883d-4114-8451-8359c3ecf4f0\") " pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:36.885964 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.885936    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99546707-eb92-4ba6-a09b-e33b2186434c-tls-certs\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:36.888319 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:36.888297    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99546707-eb92-4ba6-a09b-e33b2186434c-tls-certs\") pod \"kube-auth-proxy-85d86d5b57-2264c\" (UID: \"99546707-eb92-4ba6-a09b-e33b2186434c\") " pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:37.126119 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:37.126084    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"
Jun 02 02:52:37.189073 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:37.189040    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e25396f-883d-4114-8451-8359c3ecf4f0-cert\") pod \"odh-model-controller-76588b95-2qq4b\" (UID: \"0e25396f-883d-4114-8451-8359c3ecf4f0\") " pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:37.191189 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:37.191163    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e25396f-883d-4114-8451-8359c3ecf4f0-cert\") pod \"odh-model-controller-76588b95-2qq4b\" (UID: \"0e25396f-883d-4114-8451-8359c3ecf4f0\") " pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:37.253556 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:37.253526    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85d86d5b57-2264c"]
Jun 02 02:52:37.255994 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:52:37.255964    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99546707_eb92_4ba6_a09b_e33b2186434c.slice/crio-34e224c7d22f29553227dc4a4c3d0d312855edfd586d1cf29b8564a2b2489076 WatchSource:0}: Error finding container 34e224c7d22f29553227dc4a4c3d0d312855edfd586d1cf29b8564a2b2489076: Status 404 returned error can't find the container with id 34e224c7d22f29553227dc4a4c3d0d312855edfd586d1cf29b8564a2b2489076
Jun 02 02:52:37.362187 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:37.362151    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:37.483652 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:37.483622    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-2qq4b"]
Jun 02 02:52:37.486130 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:52:37.486097    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e25396f_883d_4114_8451_8359c3ecf4f0.slice/crio-d8896d57bd45566edadcab0743c18387e26d950dcd27f1d0685df49a0175ff67 WatchSource:0}: Error finding container d8896d57bd45566edadcab0743c18387e26d950dcd27f1d0685df49a0175ff67: Status 404 returned error can't find the container with id d8896d57bd45566edadcab0743c18387e26d950dcd27f1d0685df49a0175ff67
Jun 02 02:52:38.257926 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:38.257871    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-2qq4b" event={"ID":"0e25396f-883d-4114-8451-8359c3ecf4f0","Type":"ContainerStarted","Data":"d8896d57bd45566edadcab0743c18387e26d950dcd27f1d0685df49a0175ff67"}
Jun 02 02:52:38.259261 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:38.259195    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c" event={"ID":"99546707-eb92-4ba6-a09b-e33b2186434c","Type":"ContainerStarted","Data":"34e224c7d22f29553227dc4a4c3d0d312855edfd586d1cf29b8564a2b2489076"}
Jun 02 02:52:42.272490 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:42.272437    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c" event={"ID":"99546707-eb92-4ba6-a09b-e33b2186434c","Type":"ContainerStarted","Data":"21956b34b8dcad0ee56529efc9cea7039a685ff8d7b070e8df5521c7f78e619b"}
Jun 02 02:52:42.273752 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:42.273723    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-2qq4b" event={"ID":"0e25396f-883d-4114-8451-8359c3ecf4f0","Type":"ContainerStarted","Data":"1c10c92d8157f98246169af67495b0247c2cc3a1069212ea431b432a990a2349"}
Jun 02 02:52:42.273883 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:42.273856    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:42.289446 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:42.289386    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85d86d5b57-2264c" podStartSLOduration=2.199900336 podStartE2EDuration="6.289373894s" podCreationTimestamp="2026-06-02 02:52:36 +0000 UTC" firstStartedPulling="2026-06-02 02:52:37.257755656 +0000 UTC m=+445.874244725" lastFinishedPulling="2026-06-02 02:52:41.347229209 +0000 UTC m=+449.963718283" observedRunningTime="2026-06-02 02:52:42.28874611 +0000 UTC m=+450.905235201" watchObservedRunningTime="2026-06-02 02:52:42.289373894 +0000 UTC m=+450.905862984"
Jun 02 02:52:42.311783 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:42.311724    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-2qq4b" podStartSLOduration=2.455688862 podStartE2EDuration="6.311707823s" podCreationTimestamp="2026-06-02 02:52:36 +0000 UTC" firstStartedPulling="2026-06-02 02:52:37.487488612 +0000 UTC m=+446.103977681" lastFinishedPulling="2026-06-02 02:52:41.343507573 +0000 UTC m=+449.959996642" observedRunningTime="2026-06-02 02:52:42.310012801 +0000 UTC m=+450.926501914" watchObservedRunningTime="2026-06-02 02:52:42.311707823 +0000 UTC m=+450.928196914"
Jun 02 02:52:44.167702 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.167665    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"]
Jun 02 02:52:44.170989 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.170969    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.174677 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.174654    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jun 02 02:52:44.179567 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.179542    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jun 02 02:52:44.179791 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.179563    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bnsbg\""
Jun 02 02:52:44.179891 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.179589    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jun 02 02:52:44.179891 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.179616    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jun 02 02:52:44.180013 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.179640    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jun 02 02:52:44.197321 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.197298    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"]
Jun 02 02:52:44.250587 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.250558    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-manager-config\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.250721 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.250604    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rzkf\" (UniqueName: \"kubernetes.io/projected/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-kube-api-access-7rzkf\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.250721 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.250633    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-cert\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.250721 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.250649    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-metrics-cert\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.351810 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.351772    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-cert\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.351810 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.351813    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-metrics-cert\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.352067 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.351860    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-manager-config\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.352067 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.351899    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzkf\" (UniqueName: \"kubernetes.io/projected/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-kube-api-access-7rzkf\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.352708 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.352675    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-manager-config\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.354596 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.354576    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-cert\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.354703 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.354622    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-metrics-cert\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.359832 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.359807    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rzkf\" (UniqueName: \"kubernetes.io/projected/93401a05-a109-4fb2-a703-5d8e4a3f3ba9-kube-api-access-7rzkf\") pod \"lws-controller-manager-774f9bf4f6-fplp6\" (UID: \"93401a05-a109-4fb2-a703-5d8e4a3f3ba9\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.484229 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.484140    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:44.613422 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:44.613400    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"]
Jun 02 02:52:44.615861 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:52:44.615827    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93401a05_a109_4fb2_a703_5d8e4a3f3ba9.slice/crio-fa6da9959707b6fa5b1b33309452e557a719496e9cc62b3ec5d12cd69ad81d92 WatchSource:0}: Error finding container fa6da9959707b6fa5b1b33309452e557a719496e9cc62b3ec5d12cd69ad81d92: Status 404 returned error can't find the container with id fa6da9959707b6fa5b1b33309452e557a719496e9cc62b3ec5d12cd69ad81d92
Jun 02 02:52:45.283612 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:45.283571    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6" event={"ID":"93401a05-a109-4fb2-a703-5d8e4a3f3ba9","Type":"ContainerStarted","Data":"fa6da9959707b6fa5b1b33309452e557a719496e9cc62b3ec5d12cd69ad81d92"}
Jun 02 02:52:47.291149 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:47.291060    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6" event={"ID":"93401a05-a109-4fb2-a703-5d8e4a3f3ba9","Type":"ContainerStarted","Data":"4f8d03b6196e4b217c378d8c2c5a05f61a0a391b2b2119ef788377d6a7509da3"}
Jun 02 02:52:47.291520 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:47.291176    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:52:47.311429 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:47.311378    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6" podStartSLOduration=0.945302217 podStartE2EDuration="3.311362662s" podCreationTimestamp="2026-06-02 02:52:44 +0000 UTC" firstStartedPulling="2026-06-02 02:52:44.617772136 +0000 UTC m=+453.234261205" lastFinishedPulling="2026-06-02 02:52:46.983832577 +0000 UTC m=+455.600321650" observedRunningTime="2026-06-02 02:52:47.309494242 +0000 UTC m=+455.925983331" watchObservedRunningTime="2026-06-02 02:52:47.311362662 +0000 UTC m=+455.927851753"
Jun 02 02:52:53.279115 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:53.279084    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-2qq4b"
Jun 02 02:52:58.297512 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:52:58.297474    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-fplp6"
Jun 02 02:54:23.749375 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.749290    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"]
Jun 02 02:54:23.752328 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.752310    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:23.754980 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.754954    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jun 02 02:54:23.755114 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.755095    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jun 02 02:54:23.755164 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.755095    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jun 02 02:54:23.756227 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.756200    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mcdtr\""
Jun 02 02:54:23.756349 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.756231    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jun 02 02:54:23.760211 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.760187    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"]
Jun 02 02:54:23.903540 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.903510    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6445b01-c28d-4fa9-af7d-7fcce384f1a4-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-mzh86\" (UID: \"f6445b01-c28d-4fa9-af7d-7fcce384f1a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:23.903738 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.903553    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjfw6\" (UniqueName: \"kubernetes.io/projected/f6445b01-c28d-4fa9-af7d-7fcce384f1a4-kube-api-access-hjfw6\") pod \"kuadrant-console-plugin-6cb54b5c86-mzh86\" (UID: \"f6445b01-c28d-4fa9-af7d-7fcce384f1a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:23.903738 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:23.903597    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6445b01-c28d-4fa9-af7d-7fcce384f1a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-mzh86\" (UID: \"f6445b01-c28d-4fa9-af7d-7fcce384f1a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:24.004640 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:24.004549    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6445b01-c28d-4fa9-af7d-7fcce384f1a4-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-mzh86\" (UID: \"f6445b01-c28d-4fa9-af7d-7fcce384f1a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:24.004640 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:24.004597    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfw6\" (UniqueName: \"kubernetes.io/projected/f6445b01-c28d-4fa9-af7d-7fcce384f1a4-kube-api-access-hjfw6\") pod \"kuadrant-console-plugin-6cb54b5c86-mzh86\" (UID: \"f6445b01-c28d-4fa9-af7d-7fcce384f1a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:24.004852 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:24.004725    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6445b01-c28d-4fa9-af7d-7fcce384f1a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-mzh86\" (UID: \"f6445b01-c28d-4fa9-af7d-7fcce384f1a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:24.005154 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:24.005133    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6445b01-c28d-4fa9-af7d-7fcce384f1a4-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-mzh86\" (UID: \"f6445b01-c28d-4fa9-af7d-7fcce384f1a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:24.007031 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:24.007011    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6445b01-c28d-4fa9-af7d-7fcce384f1a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-mzh86\" (UID: \"f6445b01-c28d-4fa9-af7d-7fcce384f1a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:24.012264 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:24.012238    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjfw6\" (UniqueName: \"kubernetes.io/projected/f6445b01-c28d-4fa9-af7d-7fcce384f1a4-kube-api-access-hjfw6\") pod \"kuadrant-console-plugin-6cb54b5c86-mzh86\" (UID: \"f6445b01-c28d-4fa9-af7d-7fcce384f1a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:24.061612 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:24.061588    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"
Jun 02 02:54:24.182368 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:24.182331    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86"]
Jun 02 02:54:24.185346 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:54:24.185322    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6445b01_c28d_4fa9_af7d_7fcce384f1a4.slice/crio-8d8351d2d00920e0c1e3011b0d4acc2658a62ec018efa6611828bcd051e43d2d WatchSource:0}: Error finding container 8d8351d2d00920e0c1e3011b0d4acc2658a62ec018efa6611828bcd051e43d2d: Status 404 returned error can't find the container with id 8d8351d2d00920e0c1e3011b0d4acc2658a62ec018efa6611828bcd051e43d2d
Jun 02 02:54:24.560523 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:24.560487    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86" event={"ID":"f6445b01-c28d-4fa9-af7d-7fcce384f1a4","Type":"ContainerStarted","Data":"8d8351d2d00920e0c1e3011b0d4acc2658a62ec018efa6611828bcd051e43d2d"}
Jun 02 02:54:47.633566 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:47.633530    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86" event={"ID":"f6445b01-c28d-4fa9-af7d-7fcce384f1a4","Type":"ContainerStarted","Data":"6ac543ea7c9ae43a4328f3ae3f7bf748242121644a15df9fb7b1b76a6d8eae58"}
Jun 02 02:54:47.650572 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:54:47.650526    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-mzh86" podStartSLOduration=2.092295178 podStartE2EDuration="24.65051212s" podCreationTimestamp="2026-06-02 02:54:23 +0000 UTC" firstStartedPulling="2026-06-02 02:54:24.186606314 +0000 UTC m=+552.803095384" lastFinishedPulling="2026-06-02 02:54:46.744823253 +0000 UTC m=+575.361312326" observedRunningTime="2026-06-02 02:54:47.650339891 +0000 UTC m=+576.266828991" watchObservedRunningTime="2026-06-02 02:54:47.65051212 +0000 UTC m=+576.267001212"
Jun 02 02:55:11.892383 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:11.892357    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 02:55:11.892910 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:11.892567    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 02:55:17.284730 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.284669    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-pj552"]
Jun 02 02:55:17.388253 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.388219    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-pj552"]
Jun 02 02:55:17.388412 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.388324    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-pj552"
Jun 02 02:55:17.391386 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.391363    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d7n9s\""
Jun 02 02:55:17.539148 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.539066    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9zb2\" (UniqueName: \"kubernetes.io/projected/593ca3ae-700c-402e-8058-81453f353532-kube-api-access-k9zb2\") pod \"authorino-f99f4b5cd-pj552\" (UID: \"593ca3ae-700c-402e-8058-81453f353532\") " pod="kuadrant-system/authorino-f99f4b5cd-pj552"
Jun 02 02:55:17.570334 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.570300    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-tkw4q"]
Jun 02 02:55:17.629409 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.629375    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-tkw4q"]
Jun 02 02:55:17.629583 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.629504    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-tkw4q"
Jun 02 02:55:17.639723 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.639695    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k9zb2\" (UniqueName: \"kubernetes.io/projected/593ca3ae-700c-402e-8058-81453f353532-kube-api-access-k9zb2\") pod \"authorino-f99f4b5cd-pj552\" (UID: \"593ca3ae-700c-402e-8058-81453f353532\") " pod="kuadrant-system/authorino-f99f4b5cd-pj552"
Jun 02 02:55:17.650368 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.650345    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9zb2\" (UniqueName: \"kubernetes.io/projected/593ca3ae-700c-402e-8058-81453f353532-kube-api-access-k9zb2\") pod \"authorino-f99f4b5cd-pj552\" (UID: \"593ca3ae-700c-402e-8058-81453f353532\") " pod="kuadrant-system/authorino-f99f4b5cd-pj552"
Jun 02 02:55:17.697774 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.697741    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-pj552"
Jun 02 02:55:17.740760 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.740730    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tll62\" (UniqueName: \"kubernetes.io/projected/fde186e3-1668-412c-809f-1df61d82565e-kube-api-access-tll62\") pod \"authorino-7498df8756-tkw4q\" (UID: \"fde186e3-1668-412c-809f-1df61d82565e\") " pod="kuadrant-system/authorino-7498df8756-tkw4q"
Jun 02 02:55:17.822148 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.822117    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-pj552"]
Jun 02 02:55:17.825653 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:55:17.825607    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod593ca3ae_700c_402e_8058_81453f353532.slice/crio-e1c8a78e2689504794e636923272273b2f67ee3e5354f1332ce3368fedd836b7 WatchSource:0}: Error finding container e1c8a78e2689504794e636923272273b2f67ee3e5354f1332ce3368fedd836b7: Status 404 returned error can't find the container with id e1c8a78e2689504794e636923272273b2f67ee3e5354f1332ce3368fedd836b7
Jun 02 02:55:17.841627 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.841605    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tll62\" (UniqueName: \"kubernetes.io/projected/fde186e3-1668-412c-809f-1df61d82565e-kube-api-access-tll62\") pod \"authorino-7498df8756-tkw4q\" (UID: \"fde186e3-1668-412c-809f-1df61d82565e\") " pod="kuadrant-system/authorino-7498df8756-tkw4q"
Jun 02 02:55:17.850333 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.850304    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tll62\" (UniqueName: \"kubernetes.io/projected/fde186e3-1668-412c-809f-1df61d82565e-kube-api-access-tll62\") pod \"authorino-7498df8756-tkw4q\" (UID: \"fde186e3-1668-412c-809f-1df61d82565e\") " pod="kuadrant-system/authorino-7498df8756-tkw4q"
Jun 02 02:55:17.938089 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:17.938060    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-tkw4q"
Jun 02 02:55:18.059748 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:18.059716    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-tkw4q"]
Jun 02 02:55:18.062625 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:55:18.062601    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfde186e3_1668_412c_809f_1df61d82565e.slice/crio-23504572e09bf3134e07e48793c011db02d68260fef6517cffb04c57201b7407 WatchSource:0}: Error finding container 23504572e09bf3134e07e48793c011db02d68260fef6517cffb04c57201b7407: Status 404 returned error can't find the container with id 23504572e09bf3134e07e48793c011db02d68260fef6517cffb04c57201b7407
Jun 02 02:55:18.722732 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:18.722691    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-tkw4q" event={"ID":"fde186e3-1668-412c-809f-1df61d82565e","Type":"ContainerStarted","Data":"23504572e09bf3134e07e48793c011db02d68260fef6517cffb04c57201b7407"}
Jun 02 02:55:18.725769 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:18.725694    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-pj552" event={"ID":"593ca3ae-700c-402e-8058-81453f353532","Type":"ContainerStarted","Data":"e1c8a78e2689504794e636923272273b2f67ee3e5354f1332ce3368fedd836b7"}
Jun 02 02:55:21.742494 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:21.742438    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-tkw4q" event={"ID":"fde186e3-1668-412c-809f-1df61d82565e","Type":"ContainerStarted","Data":"235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f"}
Jun 02 02:55:21.743835 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:21.743812    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-pj552" event={"ID":"593ca3ae-700c-402e-8058-81453f353532","Type":"ContainerStarted","Data":"7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837"}
Jun 02 02:55:21.757527 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:21.757484    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-tkw4q" podStartSLOduration=1.56409172 podStartE2EDuration="4.757439627s" podCreationTimestamp="2026-06-02 02:55:17 +0000 UTC" firstStartedPulling="2026-06-02 02:55:18.063893465 +0000 UTC m=+606.680382533" lastFinishedPulling="2026-06-02 02:55:21.257241367 +0000 UTC m=+609.873730440" observedRunningTime="2026-06-02 02:55:21.756190857 +0000 UTC m=+610.372679950" watchObservedRunningTime="2026-06-02 02:55:21.757439627 +0000 UTC m=+610.373928718"
Jun 02 02:55:21.770985 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:21.770937    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-pj552" podStartSLOduration=1.350219994 podStartE2EDuration="4.770921535s" podCreationTimestamp="2026-06-02 02:55:17 +0000 UTC" firstStartedPulling="2026-06-02 02:55:17.826570374 +0000 UTC m=+606.443059443" lastFinishedPulling="2026-06-02 02:55:21.247271911 +0000 UTC m=+609.863760984" observedRunningTime="2026-06-02 02:55:21.770044451 +0000 UTC m=+610.386533544" watchObservedRunningTime="2026-06-02 02:55:21.770921535 +0000 UTC m=+610.387410626"
Jun 02 02:55:21.793218 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:21.793181    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-pj552"]
Jun 02 02:55:23.750358 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:23.750295    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-pj552" podUID="593ca3ae-700c-402e-8058-81453f353532" containerName="authorino" containerID="cri-o://7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837" gracePeriod=30
Jun 02 02:55:23.992833 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:23.992808    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-pj552"
Jun 02 02:55:24.096219 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.096188    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9zb2\" (UniqueName: \"kubernetes.io/projected/593ca3ae-700c-402e-8058-81453f353532-kube-api-access-k9zb2\") pod \"593ca3ae-700c-402e-8058-81453f353532\" (UID: \"593ca3ae-700c-402e-8058-81453f353532\") "
Jun 02 02:55:24.098363 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.098330    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/593ca3ae-700c-402e-8058-81453f353532-kube-api-access-k9zb2" (OuterVolumeSpecName: "kube-api-access-k9zb2") pod "593ca3ae-700c-402e-8058-81453f353532" (UID: "593ca3ae-700c-402e-8058-81453f353532"). InnerVolumeSpecName "kube-api-access-k9zb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 02:55:24.197669 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.197635    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9zb2\" (UniqueName: \"kubernetes.io/projected/593ca3ae-700c-402e-8058-81453f353532-kube-api-access-k9zb2\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:55:24.753901 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.753864    2554 generic.go:358] "Generic (PLEG): container finished" podID="593ca3ae-700c-402e-8058-81453f353532" containerID="7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837" exitCode=0
Jun 02 02:55:24.754398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.753911    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-pj552" event={"ID":"593ca3ae-700c-402e-8058-81453f353532","Type":"ContainerDied","Data":"7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837"}
Jun 02 02:55:24.754398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.753913    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-pj552"
Jun 02 02:55:24.754398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.753945    2554 scope.go:117] "RemoveContainer" containerID="7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837"
Jun 02 02:55:24.754398 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.753934    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-pj552" event={"ID":"593ca3ae-700c-402e-8058-81453f353532","Type":"ContainerDied","Data":"e1c8a78e2689504794e636923272273b2f67ee3e5354f1332ce3368fedd836b7"}
Jun 02 02:55:24.762334 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.762146    2554 scope.go:117] "RemoveContainer" containerID="7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837"
Jun 02 02:55:24.762529 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:55:24.762500    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837\": container with ID starting with 7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837 not found: ID does not exist" containerID="7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837"
Jun 02 02:55:24.762627 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.762542    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837"} err="failed to get container status \"7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837\": rpc error: code = NotFound desc = could not find container \"7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837\": container with ID starting with 7a1293843d66b59e9e7ba89aac499ac65beddb8b1efdad3074e41bb84906c837 not found: ID does not exist"
Jun 02 02:55:24.778801 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.778774    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-pj552"]
Jun 02 02:55:24.784261 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:24.784239    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-pj552"]
Jun 02 02:55:26.000327 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:55:26.000288    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="593ca3ae-700c-402e-8058-81453f353532" path="/var/lib/kubelet/pods/593ca3ae-700c-402e-8058-81453f353532/volumes"
Jun 02 02:56:07.557292 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.557256    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 02 02:56:07.557763 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.557533    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="593ca3ae-700c-402e-8058-81453f353532" containerName="authorino"
Jun 02 02:56:07.557763 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.557545    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="593ca3ae-700c-402e-8058-81453f353532" containerName="authorino"
Jun 02 02:56:07.557763 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.557601    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="593ca3ae-700c-402e-8058-81453f353532" containerName="authorino"
Jun 02 02:56:07.560209 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.560193    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:56:07.562972 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.562946    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
Jun 02 02:56:07.563087 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.563017    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\""
Jun 02 02:56:07.564105 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.564088    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-l68qb\""
Jun 02 02:56:07.564169 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.564103    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\""
Jun 02 02:56:07.569615 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.569594    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 02 02:56:07.616366 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.616330    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwx7j\" (UniqueName: \"kubernetes.io/projected/f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891-kube-api-access-mwx7j\") pod \"maas-keycloak-0\" (UID: \"f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891\") " pod="keycloak-system/maas-keycloak-0"
Jun 02 02:56:07.717194 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.717155    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mwx7j\" (UniqueName: \"kubernetes.io/projected/f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891-kube-api-access-mwx7j\") pod \"maas-keycloak-0\" (UID: \"f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891\") " pod="keycloak-system/maas-keycloak-0"
Jun 02 02:56:07.725410 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.725374    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwx7j\" (UniqueName: \"kubernetes.io/projected/f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891-kube-api-access-mwx7j\") pod \"maas-keycloak-0\" (UID: \"f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891\") " pod="keycloak-system/maas-keycloak-0"
Jun 02 02:56:07.869823 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.869741    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:56:07.990611 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:07.990587    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 02 02:56:07.993293 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:56:07.993268    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b37ba8_ef5e_44d6_b11d_a8c5d27fa891.slice/crio-8f3ed1fe2fd7a4b0b7224254d2211bd515cf2cda716e5449371523c781560572 WatchSource:0}: Error finding container 8f3ed1fe2fd7a4b0b7224254d2211bd515cf2cda716e5449371523c781560572: Status 404 returned error can't find the container with id 8f3ed1fe2fd7a4b0b7224254d2211bd515cf2cda716e5449371523c781560572
Jun 02 02:56:08.890221 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:08.890185    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891","Type":"ContainerStarted","Data":"8f3ed1fe2fd7a4b0b7224254d2211bd515cf2cda716e5449371523c781560572"}
Jun 02 02:56:13.113050 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:13.113028    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
Jun 02 02:56:13.908209 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:13.908157    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891","Type":"ContainerStarted","Data":"d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2"}
Jun 02 02:56:13.926623 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:13.926565    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.810719711 podStartE2EDuration="6.926546296s" podCreationTimestamp="2026-06-02 02:56:07 +0000 UTC" firstStartedPulling="2026-06-02 02:56:07.99455925 +0000 UTC m=+656.611048319" lastFinishedPulling="2026-06-02 02:56:13.110385821 +0000 UTC m=+661.726874904" observedRunningTime="2026-06-02 02:56:13.925664265 +0000 UTC m=+662.542153357" watchObservedRunningTime="2026-06-02 02:56:13.926546296 +0000 UTC m=+662.543035387"
Jun 02 02:56:14.870128 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:14.870076    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:56:14.871898 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:14.871850    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:15.870746 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:15.870696    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:16.870948 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:16.870896    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:17.870799 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:17.870760    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:56:17.871266 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:17.871221    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:18.870546 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:18.870500    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:19.871259 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:19.871209    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:20.871263 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:20.871216    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:21.870580 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:21.870530    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:22.870930 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:22.870879    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:23.871125 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:23.871078    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:24.870698 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:24.870646    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:25.870391 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:25.870336    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused"
Jun 02 02:56:26.991535 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:26.991490    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:56:27.008684 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:27.008639    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 02:56:36.997933 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:36.997896    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:56:37.868781 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:37.868746    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-774xn"]
Jun 02 02:56:37.874362 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:37.874340    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-774xn"
Jun 02 02:56:37.877287 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:37.877259    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-774xn"]
Jun 02 02:56:37.894257 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:37.894232    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th4df\" (UniqueName: \"kubernetes.io/projected/6ff0efdf-c9c6-428e-baaf-97fc26976515-kube-api-access-th4df\") pod \"authorino-8b475cf9f-774xn\" (UID: \"6ff0efdf-c9c6-428e-baaf-97fc26976515\") " pod="kuadrant-system/authorino-8b475cf9f-774xn"
Jun 02 02:56:37.995004 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:37.994974    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-th4df\" (UniqueName: \"kubernetes.io/projected/6ff0efdf-c9c6-428e-baaf-97fc26976515-kube-api-access-th4df\") pod \"authorino-8b475cf9f-774xn\" (UID: \"6ff0efdf-c9c6-428e-baaf-97fc26976515\") " pod="kuadrant-system/authorino-8b475cf9f-774xn"
Jun 02 02:56:38.002521 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.002490    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th4df\" (UniqueName: \"kubernetes.io/projected/6ff0efdf-c9c6-428e-baaf-97fc26976515-kube-api-access-th4df\") pod \"authorino-8b475cf9f-774xn\" (UID: \"6ff0efdf-c9c6-428e-baaf-97fc26976515\") " pod="kuadrant-system/authorino-8b475cf9f-774xn"
Jun 02 02:56:38.108224 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.108193    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-774xn"]
Jun 02 02:56:38.108383 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.108373    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-774xn"
Jun 02 02:56:38.133747 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.133687    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b6d6f89bd-2hx69"]
Jun 02 02:56:38.136205 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.136188    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6d6f89bd-2hx69"
Jun 02 02:56:38.143392 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.143235    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b6d6f89bd-2hx69"]
Jun 02 02:56:38.197071 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.196992    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb6rw\" (UniqueName: \"kubernetes.io/projected/b82adf1c-b067-45c7-bf86-a98032707a78-kube-api-access-xb6rw\") pod \"authorino-7b6d6f89bd-2hx69\" (UID: \"b82adf1c-b067-45c7-bf86-a98032707a78\") " pod="kuadrant-system/authorino-7b6d6f89bd-2hx69"
Jun 02 02:56:38.236561 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.236490    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-774xn"]
Jun 02 02:56:38.239375 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:56:38.239350    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ff0efdf_c9c6_428e_baaf_97fc26976515.slice/crio-8a42b0dc5b1bb8fcc6ba9250e49731076f7ac5d50b33dbd5acfb1f481a448f94 WatchSource:0}: Error finding container 8a42b0dc5b1bb8fcc6ba9250e49731076f7ac5d50b33dbd5acfb1f481a448f94: Status 404 returned error can't find the container with id 8a42b0dc5b1bb8fcc6ba9250e49731076f7ac5d50b33dbd5acfb1f481a448f94
Jun 02 02:56:38.244326 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.244304    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b6d6f89bd-2hx69"]
Jun 02 02:56:38.244622 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:56:38.244542    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xb6rw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7b6d6f89bd-2hx69" podUID="b82adf1c-b067-45c7-bf86-a98032707a78"
Jun 02 02:56:38.271539 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.271514    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-777bbc645b-892pr"]
Jun 02 02:56:38.273651 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.273636    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:56:38.276122 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.276101    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jun 02 02:56:38.282022 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.282001    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-777bbc645b-892pr"]
Jun 02 02:56:38.298197 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.298177    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2bdd5f44-a56a-4567-9028-e005885b2b2d-tls-cert\") pod \"authorino-777bbc645b-892pr\" (UID: \"2bdd5f44-a56a-4567-9028-e005885b2b2d\") " pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:56:38.298286 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.298229    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm97x\" (UniqueName: \"kubernetes.io/projected/2bdd5f44-a56a-4567-9028-e005885b2b2d-kube-api-access-cm97x\") pod \"authorino-777bbc645b-892pr\" (UID: \"2bdd5f44-a56a-4567-9028-e005885b2b2d\") " pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:56:38.298286 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.298271    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xb6rw\" (UniqueName: \"kubernetes.io/projected/b82adf1c-b067-45c7-bf86-a98032707a78-kube-api-access-xb6rw\") pod \"authorino-7b6d6f89bd-2hx69\" (UID: \"b82adf1c-b067-45c7-bf86-a98032707a78\") " pod="kuadrant-system/authorino-7b6d6f89bd-2hx69"
Jun 02 02:56:38.306094 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.306070    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb6rw\" (UniqueName: \"kubernetes.io/projected/b82adf1c-b067-45c7-bf86-a98032707a78-kube-api-access-xb6rw\") pod \"authorino-7b6d6f89bd-2hx69\" (UID: \"b82adf1c-b067-45c7-bf86-a98032707a78\") " pod="kuadrant-system/authorino-7b6d6f89bd-2hx69"
Jun 02 02:56:38.399158 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.399088    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2bdd5f44-a56a-4567-9028-e005885b2b2d-tls-cert\") pod \"authorino-777bbc645b-892pr\" (UID: \"2bdd5f44-a56a-4567-9028-e005885b2b2d\") " pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:56:38.399158 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.399138    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cm97x\" (UniqueName: \"kubernetes.io/projected/2bdd5f44-a56a-4567-9028-e005885b2b2d-kube-api-access-cm97x\") pod \"authorino-777bbc645b-892pr\" (UID: \"2bdd5f44-a56a-4567-9028-e005885b2b2d\") " pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:56:38.401375 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.401358    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2bdd5f44-a56a-4567-9028-e005885b2b2d-tls-cert\") pod \"authorino-777bbc645b-892pr\" (UID: \"2bdd5f44-a56a-4567-9028-e005885b2b2d\") " pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:56:38.406838 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.406820    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm97x\" (UniqueName: \"kubernetes.io/projected/2bdd5f44-a56a-4567-9028-e005885b2b2d-kube-api-access-cm97x\") pod \"authorino-777bbc645b-892pr\" (UID: \"2bdd5f44-a56a-4567-9028-e005885b2b2d\") " pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:56:38.583118 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.583095    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:56:38.710688 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.710661    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-777bbc645b-892pr"]
Jun 02 02:56:38.712877 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:56:38.712843    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bdd5f44_a56a_4567_9028_e005885b2b2d.slice/crio-97eac6969b91a78dfdb8665a7db8322adab15c61bb3b0e312d79d1419b67297c WatchSource:0}: Error finding container 97eac6969b91a78dfdb8665a7db8322adab15c61bb3b0e312d79d1419b67297c: Status 404 returned error can't find the container with id 97eac6969b91a78dfdb8665a7db8322adab15c61bb3b0e312d79d1419b67297c
Jun 02 02:56:38.998032 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.997946    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-777bbc645b-892pr" event={"ID":"2bdd5f44-a56a-4567-9028-e005885b2b2d","Type":"ContainerStarted","Data":"97eac6969b91a78dfdb8665a7db8322adab15c61bb3b0e312d79d1419b67297c"}
Jun 02 02:56:38.999125 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.999101    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6d6f89bd-2hx69"
Jun 02 02:56:38.999237 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.999140    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-774xn" podUID="6ff0efdf-c9c6-428e-baaf-97fc26976515" containerName="authorino" containerID="cri-o://214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f" gracePeriod=30
Jun 02 02:56:38.999237 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.999107    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-774xn" event={"ID":"6ff0efdf-c9c6-428e-baaf-97fc26976515","Type":"ContainerStarted","Data":"214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f"}
Jun 02 02:56:38.999325 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:38.999239    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-774xn" event={"ID":"6ff0efdf-c9c6-428e-baaf-97fc26976515","Type":"ContainerStarted","Data":"8a42b0dc5b1bb8fcc6ba9250e49731076f7ac5d50b33dbd5acfb1f481a448f94"}
Jun 02 02:56:39.004260 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:39.004236    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6d6f89bd-2hx69"
Jun 02 02:56:39.016086 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:39.016036    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-774xn" podStartSLOduration=1.715186946 podStartE2EDuration="2.016006701s" podCreationTimestamp="2026-06-02 02:56:37 +0000 UTC" firstStartedPulling="2026-06-02 02:56:38.240571072 +0000 UTC m=+686.857060140" lastFinishedPulling="2026-06-02 02:56:38.541390813 +0000 UTC m=+687.157879895" observedRunningTime="2026-06-02 02:56:39.015329864 +0000 UTC m=+687.631818954" watchObservedRunningTime="2026-06-02 02:56:39.016006701 +0000 UTC m=+687.632495795"
Jun 02 02:56:39.105057 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:39.105029    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb6rw\" (UniqueName: \"kubernetes.io/projected/b82adf1c-b067-45c7-bf86-a98032707a78-kube-api-access-xb6rw\") pod \"b82adf1c-b067-45c7-bf86-a98032707a78\" (UID: \"b82adf1c-b067-45c7-bf86-a98032707a78\") "
Jun 02 02:56:39.106801 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:39.106772    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b82adf1c-b067-45c7-bf86-a98032707a78-kube-api-access-xb6rw" (OuterVolumeSpecName: "kube-api-access-xb6rw") pod "b82adf1c-b067-45c7-bf86-a98032707a78" (UID: "b82adf1c-b067-45c7-bf86-a98032707a78"). InnerVolumeSpecName "kube-api-access-xb6rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 02:56:39.206153 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:39.206114    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb6rw\" (UniqueName: \"kubernetes.io/projected/b82adf1c-b067-45c7-bf86-a98032707a78-kube-api-access-xb6rw\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:56:39.224835 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:39.224807    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-774xn"
Jun 02 02:56:39.306417 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:39.306383    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-th4df\" (UniqueName: \"kubernetes.io/projected/6ff0efdf-c9c6-428e-baaf-97fc26976515-kube-api-access-th4df\") pod \"6ff0efdf-c9c6-428e-baaf-97fc26976515\" (UID: \"6ff0efdf-c9c6-428e-baaf-97fc26976515\") "
Jun 02 02:56:39.308377 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:39.308356    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff0efdf-c9c6-428e-baaf-97fc26976515-kube-api-access-th4df" (OuterVolumeSpecName: "kube-api-access-th4df") pod "6ff0efdf-c9c6-428e-baaf-97fc26976515" (UID: "6ff0efdf-c9c6-428e-baaf-97fc26976515"). InnerVolumeSpecName "kube-api-access-th4df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 02:56:39.407787 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:39.407760    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th4df\" (UniqueName: \"kubernetes.io/projected/6ff0efdf-c9c6-428e-baaf-97fc26976515-kube-api-access-th4df\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:56:40.009620 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.009575    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-777bbc645b-892pr" event={"ID":"2bdd5f44-a56a-4567-9028-e005885b2b2d","Type":"ContainerStarted","Data":"7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db"}
Jun 02 02:56:40.011104 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.011079    2554 generic.go:358] "Generic (PLEG): container finished" podID="6ff0efdf-c9c6-428e-baaf-97fc26976515" containerID="214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f" exitCode=0
Jun 02 02:56:40.011180 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.011138    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-774xn"
Jun 02 02:56:40.011180 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.011151    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6d6f89bd-2hx69"
Jun 02 02:56:40.011180 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.011153    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-774xn" event={"ID":"6ff0efdf-c9c6-428e-baaf-97fc26976515","Type":"ContainerDied","Data":"214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f"}
Jun 02 02:56:40.011322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.011184    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-774xn" event={"ID":"6ff0efdf-c9c6-428e-baaf-97fc26976515","Type":"ContainerDied","Data":"8a42b0dc5b1bb8fcc6ba9250e49731076f7ac5d50b33dbd5acfb1f481a448f94"}
Jun 02 02:56:40.011322 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.011200    2554 scope.go:117] "RemoveContainer" containerID="214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f"
Jun 02 02:56:40.022315 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.022299    2554 scope.go:117] "RemoveContainer" containerID="214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f"
Jun 02 02:56:40.022579 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:56:40.022561    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f\": container with ID starting with 214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f not found: ID does not exist" containerID="214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f"
Jun 02 02:56:40.022628 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.022588    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f"} err="failed to get container status \"214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f\": rpc error: code = NotFound desc = could not find container \"214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f\": container with ID starting with 214e05ac5aafcfca3a8fb0fd06eff73eea36e1751110d4d70e686c86c660b10f not found: ID does not exist"
Jun 02 02:56:40.026949 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.026912    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-777bbc645b-892pr" podStartSLOduration=1.663989683 podStartE2EDuration="2.026900949s" podCreationTimestamp="2026-06-02 02:56:38 +0000 UTC" firstStartedPulling="2026-06-02 02:56:38.714152205 +0000 UTC m=+687.330641274" lastFinishedPulling="2026-06-02 02:56:39.077063455 +0000 UTC m=+687.693552540" observedRunningTime="2026-06-02 02:56:40.025980168 +0000 UTC m=+688.642469258" watchObservedRunningTime="2026-06-02 02:56:40.026900949 +0000 UTC m=+688.643390042"
Jun 02 02:56:40.040626 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.040585    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-774xn"]
Jun 02 02:56:40.042883 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.042857    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-774xn"]
Jun 02 02:56:40.051405 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.051385    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-tkw4q"]
Jun 02 02:56:40.051605 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.051576    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-tkw4q" podUID="fde186e3-1668-412c-809f-1df61d82565e" containerName="authorino" containerID="cri-o://235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f" gracePeriod=30
Jun 02 02:56:40.065982 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.065960    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b6d6f89bd-2hx69"]
Jun 02 02:56:40.069624 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.069604    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b6d6f89bd-2hx69"]
Jun 02 02:56:40.296550 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.296527    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-tkw4q"
Jun 02 02:56:40.315548 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.315522    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tll62\" (UniqueName: \"kubernetes.io/projected/fde186e3-1668-412c-809f-1df61d82565e-kube-api-access-tll62\") pod \"fde186e3-1668-412c-809f-1df61d82565e\" (UID: \"fde186e3-1668-412c-809f-1df61d82565e\") "
Jun 02 02:56:40.317906 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.317881    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde186e3-1668-412c-809f-1df61d82565e-kube-api-access-tll62" (OuterVolumeSpecName: "kube-api-access-tll62") pod "fde186e3-1668-412c-809f-1df61d82565e" (UID: "fde186e3-1668-412c-809f-1df61d82565e"). InnerVolumeSpecName "kube-api-access-tll62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 02:56:40.416552 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:40.416521    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tll62\" (UniqueName: \"kubernetes.io/projected/fde186e3-1668-412c-809f-1df61d82565e-kube-api-access-tll62\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:56:41.014981 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:41.014947    2554 generic.go:358] "Generic (PLEG): container finished" podID="fde186e3-1668-412c-809f-1df61d82565e" containerID="235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f" exitCode=0
Jun 02 02:56:41.015488 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:41.015000    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-tkw4q"
Jun 02 02:56:41.015488 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:41.015036    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-tkw4q" event={"ID":"fde186e3-1668-412c-809f-1df61d82565e","Type":"ContainerDied","Data":"235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f"}
Jun 02 02:56:41.015488 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:41.015072    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-tkw4q" event={"ID":"fde186e3-1668-412c-809f-1df61d82565e","Type":"ContainerDied","Data":"23504572e09bf3134e07e48793c011db02d68260fef6517cffb04c57201b7407"}
Jun 02 02:56:41.015488 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:41.015088    2554 scope.go:117] "RemoveContainer" containerID="235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f"
Jun 02 02:56:41.022991 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:41.022956    2554 scope.go:117] "RemoveContainer" containerID="235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f"
Jun 02 02:56:41.023216 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:56:41.023196    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f\": container with ID starting with 235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f not found: ID does not exist" containerID="235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f"
Jun 02 02:56:41.023269 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:41.023224    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f"} err="failed to get container status \"235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f\": rpc error: code = NotFound desc = could not find container \"235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f\": container with ID starting with 235b22a51ad88e329213117ee30bfdb51f1b82c4b0a86c6a95fac2cfe0aa071f not found: ID does not exist"
Jun 02 02:56:41.034778 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:41.034754    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-tkw4q"]
Jun 02 02:56:41.037334 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:41.037313    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-tkw4q"]
Jun 02 02:56:42.002382 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:42.002338    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ff0efdf-c9c6-428e-baaf-97fc26976515" path="/var/lib/kubelet/pods/6ff0efdf-c9c6-428e-baaf-97fc26976515/volumes"
Jun 02 02:56:42.002779 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:42.002759    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b82adf1c-b067-45c7-bf86-a98032707a78" path="/var/lib/kubelet/pods/b82adf1c-b067-45c7-bf86-a98032707a78/volumes"
Jun 02 02:56:42.003003 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:42.002987    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fde186e3-1668-412c-809f-1df61d82565e" path="/var/lib/kubelet/pods/fde186e3-1668-412c-809f-1df61d82565e/volumes"
Jun 02 02:56:43.140028 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.139988    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67f5f7446-hlqbv"]
Jun 02 02:56:43.140639 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.140263    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fde186e3-1668-412c-809f-1df61d82565e" containerName="authorino"
Jun 02 02:56:43.140639 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.140274    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="fde186e3-1668-412c-809f-1df61d82565e" containerName="authorino"
Jun 02 02:56:43.140639 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.140293    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff0efdf-c9c6-428e-baaf-97fc26976515" containerName="authorino"
Jun 02 02:56:43.140639 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.140298    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff0efdf-c9c6-428e-baaf-97fc26976515" containerName="authorino"
Jun 02 02:56:43.140639 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.140363    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="fde186e3-1668-412c-809f-1df61d82565e" containerName="authorino"
Jun 02 02:56:43.140639 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.140383    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="6ff0efdf-c9c6-428e-baaf-97fc26976515" containerName="authorino"
Jun 02 02:56:43.142791 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.142775    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67f5f7446-hlqbv"
Jun 02 02:56:43.145560 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.145538    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jun 02 02:56:43.145706 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.145691    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kqxff\""
Jun 02 02:56:43.150428 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.150409    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67f5f7446-hlqbv"]
Jun 02 02:56:43.237114 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.237082    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qdrh\" (UniqueName: \"kubernetes.io/projected/d97868dd-dfb2-4803-a606-035a224a1caf-kube-api-access-2qdrh\") pod \"maas-controller-67f5f7446-hlqbv\" (UID: \"d97868dd-dfb2-4803-a606-035a224a1caf\") " pod="opendatahub/maas-controller-67f5f7446-hlqbv"
Jun 02 02:56:43.307115 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.307080    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86dc8749c7-8hrtr"]
Jun 02 02:56:43.309116 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.309099    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc8749c7-8hrtr"
Jun 02 02:56:43.317973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.317949    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86dc8749c7-8hrtr"]
Jun 02 02:56:43.338316 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.338284    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2qdrh\" (UniqueName: \"kubernetes.io/projected/d97868dd-dfb2-4803-a606-035a224a1caf-kube-api-access-2qdrh\") pod \"maas-controller-67f5f7446-hlqbv\" (UID: \"d97868dd-dfb2-4803-a606-035a224a1caf\") " pod="opendatahub/maas-controller-67f5f7446-hlqbv"
Jun 02 02:56:43.338478 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.338323    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzqnw\" (UniqueName: \"kubernetes.io/projected/04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc-kube-api-access-pzqnw\") pod \"maas-controller-86dc8749c7-8hrtr\" (UID: \"04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc\") " pod="opendatahub/maas-controller-86dc8749c7-8hrtr"
Jun 02 02:56:43.345734 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.345716    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qdrh\" (UniqueName: \"kubernetes.io/projected/d97868dd-dfb2-4803-a606-035a224a1caf-kube-api-access-2qdrh\") pod \"maas-controller-67f5f7446-hlqbv\" (UID: \"d97868dd-dfb2-4803-a606-035a224a1caf\") " pod="opendatahub/maas-controller-67f5f7446-hlqbv"
Jun 02 02:56:43.438966 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.438885    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pzqnw\" (UniqueName: \"kubernetes.io/projected/04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc-kube-api-access-pzqnw\") pod \"maas-controller-86dc8749c7-8hrtr\" (UID: \"04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc\") " pod="opendatahub/maas-controller-86dc8749c7-8hrtr"
Jun 02 02:56:43.447077 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.447056    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzqnw\" (UniqueName: \"kubernetes.io/projected/04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc-kube-api-access-pzqnw\") pod \"maas-controller-86dc8749c7-8hrtr\" (UID: \"04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc\") " pod="opendatahub/maas-controller-86dc8749c7-8hrtr"
Jun 02 02:56:43.453460 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.453437    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67f5f7446-hlqbv"
Jun 02 02:56:43.589785 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.589762    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67f5f7446-hlqbv"]
Jun 02 02:56:43.592027 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:56:43.591999    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd97868dd_dfb2_4803_a606_035a224a1caf.slice/crio-b3be0dcfd8cda4a362c766db4432a79e85f2125c7a475ef350b953abc5075910 WatchSource:0}: Error finding container b3be0dcfd8cda4a362c766db4432a79e85f2125c7a475ef350b953abc5075910: Status 404 returned error can't find the container with id b3be0dcfd8cda4a362c766db4432a79e85f2125c7a475ef350b953abc5075910
Jun 02 02:56:43.620107 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.620080    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc8749c7-8hrtr"
Jun 02 02:56:43.739317 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:43.739180    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86dc8749c7-8hrtr"]
Jun 02 02:56:43.741760 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:56:43.741730    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04036ab1_cbdc_40f4_a3e2_f4d87ccd90dc.slice/crio-cff1ca3021a5b4f5746d9bfab5da22368c3c1c1bde5e19eb53c62f6fee5c248e WatchSource:0}: Error finding container cff1ca3021a5b4f5746d9bfab5da22368c3c1c1bde5e19eb53c62f6fee5c248e: Status 404 returned error can't find the container with id cff1ca3021a5b4f5746d9bfab5da22368c3c1c1bde5e19eb53c62f6fee5c248e
Jun 02 02:56:44.026865 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:44.026765    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc8749c7-8hrtr" event={"ID":"04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc","Type":"ContainerStarted","Data":"cff1ca3021a5b4f5746d9bfab5da22368c3c1c1bde5e19eb53c62f6fee5c248e"}
Jun 02 02:56:44.027626 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:44.027604    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67f5f7446-hlqbv" event={"ID":"d97868dd-dfb2-4803-a606-035a224a1caf","Type":"ContainerStarted","Data":"b3be0dcfd8cda4a362c766db4432a79e85f2125c7a475ef350b953abc5075910"}
Jun 02 02:56:47.039073 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:47.039030    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc8749c7-8hrtr" event={"ID":"04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc","Type":"ContainerStarted","Data":"1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa"}
Jun 02 02:56:47.039584 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:47.039154    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86dc8749c7-8hrtr"
Jun 02 02:56:47.040321 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:47.040297    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67f5f7446-hlqbv" event={"ID":"d97868dd-dfb2-4803-a606-035a224a1caf","Type":"ContainerStarted","Data":"312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a"}
Jun 02 02:56:47.040402 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:47.040346    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67f5f7446-hlqbv"
Jun 02 02:56:47.061497 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:47.061435    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-86dc8749c7-8hrtr" podStartSLOduration=1.633312665 podStartE2EDuration="4.06142311s" podCreationTimestamp="2026-06-02 02:56:43 +0000 UTC" firstStartedPulling="2026-06-02 02:56:43.74309635 +0000 UTC m=+692.359585418" lastFinishedPulling="2026-06-02 02:56:46.171206795 +0000 UTC m=+694.787695863" observedRunningTime="2026-06-02 02:56:47.060080129 +0000 UTC m=+695.676569219" watchObservedRunningTime="2026-06-02 02:56:47.06142311 +0000 UTC m=+695.677912194"
Jun 02 02:56:47.076712 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:47.076664    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-67f5f7446-hlqbv" podStartSLOduration=1.502570185 podStartE2EDuration="4.07665208s" podCreationTimestamp="2026-06-02 02:56:43 +0000 UTC" firstStartedPulling="2026-06-02 02:56:43.593299891 +0000 UTC m=+692.209788964" lastFinishedPulling="2026-06-02 02:56:46.167381786 +0000 UTC m=+694.783870859" observedRunningTime="2026-06-02 02:56:47.074853958 +0000 UTC m=+695.691343049" watchObservedRunningTime="2026-06-02 02:56:47.07665208 +0000 UTC m=+695.693141170"
Jun 02 02:56:58.050001 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:58.049970    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86dc8749c7-8hrtr"
Jun 02 02:56:58.050389 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:58.050056    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67f5f7446-hlqbv"
Jun 02 02:56:58.108843 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:58.108804    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67f5f7446-hlqbv"]
Jun 02 02:56:58.109065 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:58.109041    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-67f5f7446-hlqbv" podUID="d97868dd-dfb2-4803-a606-035a224a1caf" containerName="manager" containerID="cri-o://312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a" gracePeriod=30
Jun 02 02:56:58.344045 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:58.344024    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67f5f7446-hlqbv"
Jun 02 02:56:58.463591 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:58.463551    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qdrh\" (UniqueName: \"kubernetes.io/projected/d97868dd-dfb2-4803-a606-035a224a1caf-kube-api-access-2qdrh\") pod \"d97868dd-dfb2-4803-a606-035a224a1caf\" (UID: \"d97868dd-dfb2-4803-a606-035a224a1caf\") "
Jun 02 02:56:58.465655 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:58.465618    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d97868dd-dfb2-4803-a606-035a224a1caf-kube-api-access-2qdrh" (OuterVolumeSpecName: "kube-api-access-2qdrh") pod "d97868dd-dfb2-4803-a606-035a224a1caf" (UID: "d97868dd-dfb2-4803-a606-035a224a1caf"). InnerVolumeSpecName "kube-api-access-2qdrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 02:56:58.564696 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:58.564654    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qdrh\" (UniqueName: \"kubernetes.io/projected/d97868dd-dfb2-4803-a606-035a224a1caf-kube-api-access-2qdrh\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:56:59.074889 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.074852    2554 generic.go:358] "Generic (PLEG): container finished" podID="d97868dd-dfb2-4803-a606-035a224a1caf" containerID="312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a" exitCode=0
Jun 02 02:56:59.075305 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.074903    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67f5f7446-hlqbv" event={"ID":"d97868dd-dfb2-4803-a606-035a224a1caf","Type":"ContainerDied","Data":"312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a"}
Jun 02 02:56:59.075305 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.074911    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67f5f7446-hlqbv"
Jun 02 02:56:59.075305 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.074929    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67f5f7446-hlqbv" event={"ID":"d97868dd-dfb2-4803-a606-035a224a1caf","Type":"ContainerDied","Data":"b3be0dcfd8cda4a362c766db4432a79e85f2125c7a475ef350b953abc5075910"}
Jun 02 02:56:59.075305 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.074945    2554 scope.go:117] "RemoveContainer" containerID="312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a"
Jun 02 02:56:59.082624 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.082600    2554 scope.go:117] "RemoveContainer" containerID="312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a"
Jun 02 02:56:59.082892 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:56:59.082873    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a\": container with ID starting with 312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a not found: ID does not exist" containerID="312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a"
Jun 02 02:56:59.082969 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.082921    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a"} err="failed to get container status \"312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a\": rpc error: code = NotFound desc = could not find container \"312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a\": container with ID starting with 312d379532b39aedf106aacf3d763d698d059eb15ca4c401003ecae13575766a not found: ID does not exist"
Jun 02 02:56:59.101436 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.101410    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67f5f7446-hlqbv"]
Jun 02 02:56:59.106765 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.106740    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67f5f7446-hlqbv"]
Jun 02 02:56:59.999854 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:56:59.999818    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d97868dd-dfb2-4803-a606-035a224a1caf" path="/var/lib/kubelet/pods/d97868dd-dfb2-4803-a606-035a224a1caf/volumes"
Jun 02 02:57:17.389278 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.389239    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5d9b6489bc-hm652"]
Jun 02 02:57:17.389739 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.389534    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d97868dd-dfb2-4803-a606-035a224a1caf" containerName="manager"
Jun 02 02:57:17.389739 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.389546    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d97868dd-dfb2-4803-a606-035a224a1caf" containerName="manager"
Jun 02 02:57:17.389739 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.389607    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="d97868dd-dfb2-4803-a606-035a224a1caf" containerName="manager"
Jun 02 02:57:17.394595 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.394574    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:17.397265 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.397245    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jun 02 02:57:17.400920 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.400897    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5d9b6489bc-hm652"]
Jun 02 02:57:17.516753 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.516722    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bda536c-adc8-4fbc-87d7-3885775be5d2-maas-api-tls\") pod \"maas-api-5d9b6489bc-hm652\" (UID: \"3bda536c-adc8-4fbc-87d7-3885775be5d2\") " pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:17.516935 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.516762    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvtb4\" (UniqueName: \"kubernetes.io/projected/3bda536c-adc8-4fbc-87d7-3885775be5d2-kube-api-access-mvtb4\") pod \"maas-api-5d9b6489bc-hm652\" (UID: \"3bda536c-adc8-4fbc-87d7-3885775be5d2\") " pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:17.617510 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.617473    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bda536c-adc8-4fbc-87d7-3885775be5d2-maas-api-tls\") pod \"maas-api-5d9b6489bc-hm652\" (UID: \"3bda536c-adc8-4fbc-87d7-3885775be5d2\") " pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:17.617679 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.617523    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mvtb4\" (UniqueName: \"kubernetes.io/projected/3bda536c-adc8-4fbc-87d7-3885775be5d2-kube-api-access-mvtb4\") pod \"maas-api-5d9b6489bc-hm652\" (UID: \"3bda536c-adc8-4fbc-87d7-3885775be5d2\") " pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:17.620073 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.620039    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bda536c-adc8-4fbc-87d7-3885775be5d2-maas-api-tls\") pod \"maas-api-5d9b6489bc-hm652\" (UID: \"3bda536c-adc8-4fbc-87d7-3885775be5d2\") " pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:17.628973 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.628945    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvtb4\" (UniqueName: \"kubernetes.io/projected/3bda536c-adc8-4fbc-87d7-3885775be5d2-kube-api-access-mvtb4\") pod \"maas-api-5d9b6489bc-hm652\" (UID: \"3bda536c-adc8-4fbc-87d7-3885775be5d2\") " pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:17.705879 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.705807    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:17.839442 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.839410    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5d9b6489bc-hm652"]
Jun 02 02:57:17.843553 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:57:17.843520    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bda536c_adc8_4fbc_87d7_3885775be5d2.slice/crio-b8c5399188afd2ebb503a7c870e2c928b4d647453a20d42c7e6883d8f98be242 WatchSource:0}: Error finding container b8c5399188afd2ebb503a7c870e2c928b4d647453a20d42c7e6883d8f98be242: Status 404 returned error can't find the container with id b8c5399188afd2ebb503a7c870e2c928b4d647453a20d42c7e6883d8f98be242
Jun 02 02:57:17.844737 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:17.844708    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 02:57:18.139649 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:18.139615    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5d9b6489bc-hm652" event={"ID":"3bda536c-adc8-4fbc-87d7-3885775be5d2","Type":"ContainerStarted","Data":"b8c5399188afd2ebb503a7c870e2c928b4d647453a20d42c7e6883d8f98be242"}
Jun 02 02:57:20.149176 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:20.149142    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5d9b6489bc-hm652" event={"ID":"3bda536c-adc8-4fbc-87d7-3885775be5d2","Type":"ContainerStarted","Data":"19dd098d63cbe3c8cee6846c924d58ceb7321514dc9b3d817f43d7573d836ab0"}
Jun 02 02:57:20.149557 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:20.149313    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:20.164707 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:20.164636    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-5d9b6489bc-hm652" podStartSLOduration=1.427913282 podStartE2EDuration="3.164611287s" podCreationTimestamp="2026-06-02 02:57:17 +0000 UTC" firstStartedPulling="2026-06-02 02:57:17.844907971 +0000 UTC m=+726.461397043" lastFinishedPulling="2026-06-02 02:57:19.58160598 +0000 UTC m=+728.198095048" observedRunningTime="2026-06-02 02:57:20.164300049 +0000 UTC m=+728.780789141" watchObservedRunningTime="2026-06-02 02:57:20.164611287 +0000 UTC m=+728.781100380"
Jun 02 02:57:26.157761 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:26.157733    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5d9b6489bc-hm652"
Jun 02 02:57:27.954328 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:27.954288    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 02 02:57:27.954716 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:27.954544    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak" containerID="cri-o://d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2" gracePeriod=30
Jun 02 02:57:30.000091 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.000068    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.019649 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.019617    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwx7j\" (UniqueName: \"kubernetes.io/projected/f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891-kube-api-access-mwx7j\") pod \"f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891\" (UID: \"f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891\") "
Jun 02 02:57:30.021789 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.021763    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891-kube-api-access-mwx7j" (OuterVolumeSpecName: "kube-api-access-mwx7j") pod "f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" (UID: "f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891"). InnerVolumeSpecName "kube-api-access-mwx7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 02:57:30.121098 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.120998    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwx7j\" (UniqueName: \"kubernetes.io/projected/f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891-kube-api-access-mwx7j\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:57:30.179150 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.179117    2554 generic.go:358] "Generic (PLEG): container finished" podID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerID="d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2" exitCode=143
Jun 02 02:57:30.179300 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.179178    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.179300 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.179196    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891","Type":"ContainerDied","Data":"d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2"}
Jun 02 02:57:30.179300 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.179236    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891","Type":"ContainerDied","Data":"8f3ed1fe2fd7a4b0b7224254d2211bd515cf2cda716e5449371523c781560572"}
Jun 02 02:57:30.179300 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.179251    2554 scope.go:117] "RemoveContainer" containerID="d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2"
Jun 02 02:57:30.187631 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.187609    2554 scope.go:117] "RemoveContainer" containerID="d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2"
Jun 02 02:57:30.187867 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:57:30.187848    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2\": container with ID starting with d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2 not found: ID does not exist" containerID="d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2"
Jun 02 02:57:30.187925 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.187875    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2"} err="failed to get container status \"d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2\": rpc error: code = NotFound desc = could not find container \"d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2\": container with ID starting with d5ff086e51f0b03e88970bd855568d468b4f7611276e29efd135b3dbc3cf50f2 not found: ID does not exist"
Jun 02 02:57:30.199548 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.199520    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 02 02:57:30.203005 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.202984    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 02 02:57:30.222589 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.222565    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 02 02:57:30.222827 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.222815    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak"
Jun 02 02:57:30.222887 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.222829    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak"
Jun 02 02:57:30.222927 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.222894    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" containerName="keycloak"
Jun 02 02:57:30.226947 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.226932    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.229809 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.229787    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\""
Jun 02 02:57:30.229904 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.229796    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
Jun 02 02:57:30.229904 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.229798    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\""
Jun 02 02:57:30.229904 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.229850    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-l68qb\""
Jun 02 02:57:30.230025 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.229798    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\""
Jun 02 02:57:30.233628 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.233610    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 02 02:57:30.322436 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.322409    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/355bfdd9-77fa-4e6c-97dd-817722aaed96-test-realms\") pod \"maas-keycloak-0\" (UID: \"355bfdd9-77fa-4e6c-97dd-817722aaed96\") " pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.322596 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.322442    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m2qc\" (UniqueName: \"kubernetes.io/projected/355bfdd9-77fa-4e6c-97dd-817722aaed96-kube-api-access-4m2qc\") pod \"maas-keycloak-0\" (UID: \"355bfdd9-77fa-4e6c-97dd-817722aaed96\") " pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.423070 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.422994    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/355bfdd9-77fa-4e6c-97dd-817722aaed96-test-realms\") pod \"maas-keycloak-0\" (UID: \"355bfdd9-77fa-4e6c-97dd-817722aaed96\") " pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.423070 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.423028    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4m2qc\" (UniqueName: \"kubernetes.io/projected/355bfdd9-77fa-4e6c-97dd-817722aaed96-kube-api-access-4m2qc\") pod \"maas-keycloak-0\" (UID: \"355bfdd9-77fa-4e6c-97dd-817722aaed96\") " pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.423692 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.423671    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/355bfdd9-77fa-4e6c-97dd-817722aaed96-test-realms\") pod \"maas-keycloak-0\" (UID: \"355bfdd9-77fa-4e6c-97dd-817722aaed96\") " pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.431584 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.431562    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m2qc\" (UniqueName: \"kubernetes.io/projected/355bfdd9-77fa-4e6c-97dd-817722aaed96-kube-api-access-4m2qc\") pod \"maas-keycloak-0\" (UID: \"355bfdd9-77fa-4e6c-97dd-817722aaed96\") " pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.538153 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.538120    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:30.660244 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:30.660212    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 02 02:57:30.663323 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:57:30.663299    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355bfdd9_77fa_4e6c_97dd_817722aaed96.slice/crio-e9e553bd406ece12a838d780b39d9fc2128bc58963f4b9cc23c850bc5b0ebf4a WatchSource:0}: Error finding container e9e553bd406ece12a838d780b39d9fc2128bc58963f4b9cc23c850bc5b0ebf4a: Status 404 returned error can't find the container with id e9e553bd406ece12a838d780b39d9fc2128bc58963f4b9cc23c850bc5b0ebf4a
Jun 02 02:57:31.188763 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:31.188721    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"355bfdd9-77fa-4e6c-97dd-817722aaed96","Type":"ContainerStarted","Data":"325b476cd8dd529b4fe5ad3422c6be008c0fb656778f37dc62c3eebeee1ddcb1"}
Jun 02 02:57:31.189157 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:31.188772    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"355bfdd9-77fa-4e6c-97dd-817722aaed96","Type":"ContainerStarted","Data":"e9e553bd406ece12a838d780b39d9fc2128bc58963f4b9cc23c850bc5b0ebf4a"}
Jun 02 02:57:31.206519 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:31.206368    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.930801052 podStartE2EDuration="1.206347886s" podCreationTimestamp="2026-06-02 02:57:30 +0000 UTC" firstStartedPulling="2026-06-02 02:57:30.664953708 +0000 UTC m=+739.281442776" lastFinishedPulling="2026-06-02 02:57:30.94050054 +0000 UTC m=+739.556989610" observedRunningTime="2026-06-02 02:57:31.205423735 +0000 UTC m=+739.821912830" watchObservedRunningTime="2026-06-02 02:57:31.206347886 +0000 UTC m=+739.822836978"
Jun 02 02:57:31.538956 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:31.538863    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:31.540234 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:31.540205    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:32.002019 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:32.001976    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891" path="/var/lib/kubelet/pods/f6b37ba8-ef5e-44d6-b11d-a8c5d27fa891/volumes"
Jun 02 02:57:32.539690 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:32.539626    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:33.539109 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:33.539063    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:34.539391 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:34.539329    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:35.539706 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:35.539657    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:36.539683 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:36.539637    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:37.543477 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:37.542602    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:38.539073 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:38.539024    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:39.539656 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:39.539601    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:40.538762 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:40.538726    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:40.539182 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:40.539151    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:41.539163 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:41.539121    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:42.539585 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:42.539530    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:43.539276 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:43.539227    2554 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused"
Jun 02 02:57:44.668830 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:44.668787    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:57:44.689685 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:44.689638    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="355bfdd9-77fa-4e6c-97dd-817722aaed96" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 02 02:57:54.675072 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:57:54.675035    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0"
Jun 02 02:58:05.364582 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.364543    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78f58594b8-2888q"]
Jun 02 02:58:05.367804 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.367781    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.370206 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.370187    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\""
Jun 02 02:58:05.375243 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.375202    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78f58594b8-2888q"]
Jun 02 02:58:05.537081 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.537044    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8543526-fa97-4d98-a117-c8407c426358-tls-cert\") pod \"authorino-78f58594b8-2888q\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") " pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.537081 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.537086    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sxgz\" (UniqueName: \"kubernetes.io/projected/a8543526-fa97-4d98-a117-c8407c426358-kube-api-access-6sxgz\") pod \"authorino-78f58594b8-2888q\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") " pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.537277 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.537132    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a8543526-fa97-4d98-a117-c8407c426358-oidc-ca\") pod \"authorino-78f58594b8-2888q\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") " pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.638591 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.638498    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8543526-fa97-4d98-a117-c8407c426358-tls-cert\") pod \"authorino-78f58594b8-2888q\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") " pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.638591 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.638551    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6sxgz\" (UniqueName: \"kubernetes.io/projected/a8543526-fa97-4d98-a117-c8407c426358-kube-api-access-6sxgz\") pod \"authorino-78f58594b8-2888q\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") " pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.638591 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.638590    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a8543526-fa97-4d98-a117-c8407c426358-oidc-ca\") pod \"authorino-78f58594b8-2888q\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") " pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.639252 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.639225    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a8543526-fa97-4d98-a117-c8407c426358-oidc-ca\") pod \"authorino-78f58594b8-2888q\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") " pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.640978 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.640952    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8543526-fa97-4d98-a117-c8407c426358-tls-cert\") pod \"authorino-78f58594b8-2888q\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") " pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.646521 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.646498    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sxgz\" (UniqueName: \"kubernetes.io/projected/a8543526-fa97-4d98-a117-c8407c426358-kube-api-access-6sxgz\") pod \"authorino-78f58594b8-2888q\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") " pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.677208 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.677172    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 02:58:05.800437 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:05.800407    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78f58594b8-2888q"]
Jun 02 02:58:05.803920 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:58:05.803885    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8543526_fa97_4d98_a117_c8407c426358.slice/crio-d5705abba0bd47ca2cc98622799b437eef812535f949db06fe5e33d095133e6e WatchSource:0}: Error finding container d5705abba0bd47ca2cc98622799b437eef812535f949db06fe5e33d095133e6e: Status 404 returned error can't find the container with id d5705abba0bd47ca2cc98622799b437eef812535f949db06fe5e33d095133e6e
Jun 02 02:58:06.312126 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.312086    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f58594b8-2888q" event={"ID":"a8543526-fa97-4d98-a117-c8407c426358","Type":"ContainerStarted","Data":"8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77"}
Jun 02 02:58:06.312126 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.312131    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f58594b8-2888q" event={"ID":"a8543526-fa97-4d98-a117-c8407c426358","Type":"ContainerStarted","Data":"d5705abba0bd47ca2cc98622799b437eef812535f949db06fe5e33d095133e6e"}
Jun 02 02:58:06.327329 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.327287    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-78f58594b8-2888q" podStartSLOduration=0.96237264 podStartE2EDuration="1.327275657s" podCreationTimestamp="2026-06-02 02:58:05 +0000 UTC" firstStartedPulling="2026-06-02 02:58:05.805096425 +0000 UTC m=+774.421585494" lastFinishedPulling="2026-06-02 02:58:06.169999435 +0000 UTC m=+774.786488511" observedRunningTime="2026-06-02 02:58:06.325706486 +0000 UTC m=+774.942195579" watchObservedRunningTime="2026-06-02 02:58:06.327275657 +0000 UTC m=+774.943764745"
Jun 02 02:58:06.349634 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.349604    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-777bbc645b-892pr"]
Jun 02 02:58:06.349903 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.349847    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-777bbc645b-892pr" podUID="2bdd5f44-a56a-4567-9028-e005885b2b2d" containerName="authorino" containerID="cri-o://7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db" gracePeriod=30
Jun 02 02:58:06.594292 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.594267    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:58:06.749639 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.749551    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm97x\" (UniqueName: \"kubernetes.io/projected/2bdd5f44-a56a-4567-9028-e005885b2b2d-kube-api-access-cm97x\") pod \"2bdd5f44-a56a-4567-9028-e005885b2b2d\" (UID: \"2bdd5f44-a56a-4567-9028-e005885b2b2d\") "
Jun 02 02:58:06.749806 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.749652    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2bdd5f44-a56a-4567-9028-e005885b2b2d-tls-cert\") pod \"2bdd5f44-a56a-4567-9028-e005885b2b2d\" (UID: \"2bdd5f44-a56a-4567-9028-e005885b2b2d\") "
Jun 02 02:58:06.751703 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.751667    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bdd5f44-a56a-4567-9028-e005885b2b2d-kube-api-access-cm97x" (OuterVolumeSpecName: "kube-api-access-cm97x") pod "2bdd5f44-a56a-4567-9028-e005885b2b2d" (UID: "2bdd5f44-a56a-4567-9028-e005885b2b2d"). InnerVolumeSpecName "kube-api-access-cm97x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 02:58:06.758860 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.758835    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bdd5f44-a56a-4567-9028-e005885b2b2d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2bdd5f44-a56a-4567-9028-e005885b2b2d" (UID: "2bdd5f44-a56a-4567-9028-e005885b2b2d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 02:58:06.850642 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.850605    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm97x\" (UniqueName: \"kubernetes.io/projected/2bdd5f44-a56a-4567-9028-e005885b2b2d-kube-api-access-cm97x\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:58:06.850642 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:06.850637    2554 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2bdd5f44-a56a-4567-9028-e005885b2b2d-tls-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 02:58:07.315799 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:07.315764    2554 generic.go:358] "Generic (PLEG): container finished" podID="2bdd5f44-a56a-4567-9028-e005885b2b2d" containerID="7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db" exitCode=0
Jun 02 02:58:07.315984 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:07.315810    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-777bbc645b-892pr"
Jun 02 02:58:07.315984 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:07.315849    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-777bbc645b-892pr" event={"ID":"2bdd5f44-a56a-4567-9028-e005885b2b2d","Type":"ContainerDied","Data":"7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db"}
Jun 02 02:58:07.315984 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:07.315884    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-777bbc645b-892pr" event={"ID":"2bdd5f44-a56a-4567-9028-e005885b2b2d","Type":"ContainerDied","Data":"97eac6969b91a78dfdb8665a7db8322adab15c61bb3b0e312d79d1419b67297c"}
Jun 02 02:58:07.315984 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:07.315902    2554 scope.go:117] "RemoveContainer" containerID="7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db"
Jun 02 02:58:07.324121 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:07.324108    2554 scope.go:117] "RemoveContainer" containerID="7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db"
Jun 02 02:58:07.324378 ip-10-0-129-101 kubenswrapper[2554]: E0602 02:58:07.324360    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db\": container with ID starting with 7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db not found: ID does not exist" containerID="7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db"
Jun 02 02:58:07.324426 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:07.324385    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db"} err="failed to get container status \"7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db\": rpc error: code = NotFound desc = could not find container \"7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db\": container with ID starting with 7974e9683228b49e45b7308a0320f901ad77a11efb60c31360643c0c8149c8db not found: ID does not exist"
Jun 02 02:58:07.335637 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:07.335614    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-777bbc645b-892pr"]
Jun 02 02:58:07.339445 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:07.339423    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-777bbc645b-892pr"]
Jun 02 02:58:08.000210 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:08.000173    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bdd5f44-a56a-4567-9028-e005885b2b2d" path="/var/lib/kubelet/pods/2bdd5f44-a56a-4567-9028-e005885b2b2d/volumes"
Jun 02 02:58:24.137074 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.137035    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"]
Jun 02 02:58:24.137669 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.137438    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bdd5f44-a56a-4567-9028-e005885b2b2d" containerName="authorino"
Jun 02 02:58:24.137669 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.137469    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bdd5f44-a56a-4567-9028-e005885b2b2d" containerName="authorino"
Jun 02 02:58:24.137669 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.137529    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="2bdd5f44-a56a-4567-9028-e005885b2b2d" containerName="authorino"
Jun 02 02:58:24.142356 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.142337    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.145015 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.144990    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jun 02 02:58:24.145129 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.144999    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-84zz9\""
Jun 02 02:58:24.146129 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.146111    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jun 02 02:58:24.146239 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.146159    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jun 02 02:58:24.150116 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.150093    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"]
Jun 02 02:58:24.168621 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.168586    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.168752 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.168626    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.168752 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.168679    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glsqq\" (UniqueName: \"kubernetes.io/projected/037ccd98-3a71-411b-a07c-09d47c42743f-kube-api-access-glsqq\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.168752 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.168735    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/037ccd98-3a71-411b-a07c-09d47c42743f-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.168851 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.168768    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.168851 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.168814    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.269536 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.269496    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.269695 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.269544    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.269695 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.269568    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-glsqq\" (UniqueName: \"kubernetes.io/projected/037ccd98-3a71-411b-a07c-09d47c42743f-kube-api-access-glsqq\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.269695 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.269613    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/037ccd98-3a71-411b-a07c-09d47c42743f-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.269695 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.269655    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.269861 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.269710    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.269916 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.269893    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.270081 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.270054    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.270142 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.270085    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.271801 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.271777    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/037ccd98-3a71-411b-a07c-09d47c42743f-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.272062 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.272044    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/037ccd98-3a71-411b-a07c-09d47c42743f-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.277124 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.277102    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glsqq\" (UniqueName: \"kubernetes.io/projected/037ccd98-3a71-411b-a07c-09d47c42743f-kube-api-access-glsqq\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-ts677\" (UID: \"037ccd98-3a71-411b-a07c-09d47c42743f\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.454104 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.454016    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:24.585423 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:24.585396    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"]
Jun 02 02:58:24.587865 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:58:24.587824    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod037ccd98_3a71_411b_a07c_09d47c42743f.slice/crio-d545733ce7eda171d47ac90e27647cae2b45151772f845515554e87aec3e50ce WatchSource:0}: Error finding container d545733ce7eda171d47ac90e27647cae2b45151772f845515554e87aec3e50ce: Status 404 returned error can't find the container with id d545733ce7eda171d47ac90e27647cae2b45151772f845515554e87aec3e50ce
Jun 02 02:58:25.375004 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:25.374964    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677" event={"ID":"037ccd98-3a71-411b-a07c-09d47c42743f","Type":"ContainerStarted","Data":"d545733ce7eda171d47ac90e27647cae2b45151772f845515554e87aec3e50ce"}
Jun 02 02:58:28.387658 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:28.387616    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677" event={"ID":"037ccd98-3a71-411b-a07c-09d47c42743f","Type":"ContainerStarted","Data":"6b1c5100442421e86af6d50265b6e0822f14aa709e06ca5d3402b1d1efeacbdd"}
Jun 02 02:58:29.392355 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:29.392319    2554 generic.go:358] "Generic (PLEG): container finished" podID="037ccd98-3a71-411b-a07c-09d47c42743f" containerID="6b1c5100442421e86af6d50265b6e0822f14aa709e06ca5d3402b1d1efeacbdd" exitCode=0
Jun 02 02:58:29.392799 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:29.392415    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677" event={"ID":"037ccd98-3a71-411b-a07c-09d47c42743f","Type":"ContainerDied","Data":"6b1c5100442421e86af6d50265b6e0822f14aa709e06ca5d3402b1d1efeacbdd"}
Jun 02 02:58:31.401013 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:31.400982    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677" event={"ID":"037ccd98-3a71-411b-a07c-09d47c42743f","Type":"ContainerStarted","Data":"10124315ab70e7246f04801672b991f4d45248f151282f065fd0e29045b36815"}
Jun 02 02:58:31.419956 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:31.419904    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677" podStartSLOduration=1.620391695 podStartE2EDuration="7.419887553s" podCreationTimestamp="2026-06-02 02:58:24 +0000 UTC" firstStartedPulling="2026-06-02 02:58:24.589515107 +0000 UTC m=+793.206004175" lastFinishedPulling="2026-06-02 02:58:30.389010965 +0000 UTC m=+799.005500033" observedRunningTime="2026-06-02 02:58:31.418769148 +0000 UTC m=+800.035258239" watchObservedRunningTime="2026-06-02 02:58:31.419887553 +0000 UTC m=+800.036376642"
Jun 02 02:58:34.455052 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:34.455013    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:34.455052 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:34.455062    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:34.467215 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:34.467191    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:35.424616 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:35.424590    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-ts677"
Jun 02 02:58:52.661312 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.661276    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"]
Jun 02 02:58:52.674821 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.674794    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"]
Jun 02 02:58:52.674957 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.674928    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.677557 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.677534    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jun 02 02:58:52.795313 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.795278    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.795508 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.795321    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpzlx\" (UniqueName: \"kubernetes.io/projected/b22eca10-be94-4010-88ee-fc1e9b0ced87-kube-api-access-vpzlx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.795508 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.795473    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.795598 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.795509    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.795598 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.795537    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.795598 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.795570    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b22eca10-be94-4010-88ee-fc1e9b0ced87-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.896514 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.896479    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.896514 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.896515    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b22eca10-be94-4010-88ee-fc1e9b0ced87-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.896718 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.896635    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.896718 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.896696    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vpzlx\" (UniqueName: \"kubernetes.io/projected/b22eca10-be94-4010-88ee-fc1e9b0ced87-kube-api-access-vpzlx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.896819 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.896800    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.896865 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.896835    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.897021 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.896994    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.897167 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.897146    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.897227 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.897164    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.898858 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.898840    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b22eca10-be94-4010-88ee-fc1e9b0ced87-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.899086 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.899066    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b22eca10-be94-4010-88ee-fc1e9b0ced87-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.904592 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.904564    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpzlx\" (UniqueName: \"kubernetes.io/projected/b22eca10-be94-4010-88ee-fc1e9b0ced87-kube-api-access-vpzlx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm\" (UID: \"b22eca10-be94-4010-88ee-fc1e9b0ced87\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:52.986312 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:52.986225    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:58:53.111784 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:53.111754    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"]
Jun 02 02:58:53.114818 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:58:53.114793    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22eca10_be94_4010_88ee_fc1e9b0ced87.slice/crio-6d8d01e9cb9d8d8ff40a1cfce017e99e6914a09f2a87f7871ed38a64e67ff914 WatchSource:0}: Error finding container 6d8d01e9cb9d8d8ff40a1cfce017e99e6914a09f2a87f7871ed38a64e67ff914: Status 404 returned error can't find the container with id 6d8d01e9cb9d8d8ff40a1cfce017e99e6914a09f2a87f7871ed38a64e67ff914
Jun 02 02:58:53.469698 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:53.469667    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm" event={"ID":"b22eca10-be94-4010-88ee-fc1e9b0ced87","Type":"ContainerStarted","Data":"128a11359203539572fa319007fa21a37ecd63ea42f082a891cb1054fefcdb2b"}
Jun 02 02:58:53.469698 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:53.469705    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm" event={"ID":"b22eca10-be94-4010-88ee-fc1e9b0ced87","Type":"ContainerStarted","Data":"6d8d01e9cb9d8d8ff40a1cfce017e99e6914a09f2a87f7871ed38a64e67ff914"}
Jun 02 02:58:54.475334 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:54.475296    2554 generic.go:358] "Generic (PLEG): container finished" podID="b22eca10-be94-4010-88ee-fc1e9b0ced87" containerID="128a11359203539572fa319007fa21a37ecd63ea42f082a891cb1054fefcdb2b" exitCode=0
Jun 02 02:58:54.475861 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:54.475369    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm" event={"ID":"b22eca10-be94-4010-88ee-fc1e9b0ced87","Type":"ContainerDied","Data":"128a11359203539572fa319007fa21a37ecd63ea42f082a891cb1054fefcdb2b"}
Jun 02 02:58:55.481170 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:55.481134    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm" event={"ID":"b22eca10-be94-4010-88ee-fc1e9b0ced87","Type":"ContainerStarted","Data":"f1182547dd6289ae2eccea6c1b11970424ed0f15f0be5dd8821a3ebd1d740071"}
Jun 02 02:58:55.498897 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:58:55.498847    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm" podStartSLOduration=3.328259481 podStartE2EDuration="3.498831087s" podCreationTimestamp="2026-06-02 02:58:52 +0000 UTC" firstStartedPulling="2026-06-02 02:58:54.475944508 +0000 UTC m=+823.092433577" lastFinishedPulling="2026-06-02 02:58:54.646516112 +0000 UTC m=+823.263005183" observedRunningTime="2026-06-02 02:58:55.49776786 +0000 UTC m=+824.114256952" watchObservedRunningTime="2026-06-02 02:58:55.498831087 +0000 UTC m=+824.115320179"
Jun 02 02:59:02.986865 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:02.986823    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:59:02.987254 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:02.987128    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:59:02.999655 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:02.999633    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:59:03.518164 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:03.518130    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm"
Jun 02 02:59:07.746817 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.746774    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"]
Jun 02 02:59:07.750583 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.750562    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.753039 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.753019    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jun 02 02:59:07.757813 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.757640    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"]
Jun 02 02:59:07.821110 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.821080    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba39174e-fd3a-4edc-8926-493fa7c4d650-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.821299 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.821121    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.821299 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.821160    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkmq7\" (UniqueName: \"kubernetes.io/projected/ba39174e-fd3a-4edc-8926-493fa7c4d650-kube-api-access-dkmq7\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.821299 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.821238    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.821299 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.821294    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.821504 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.821342    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.922615 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.922573    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba39174e-fd3a-4edc-8926-493fa7c4d650-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.922807 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.922625    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.922807 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.922664    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dkmq7\" (UniqueName: \"kubernetes.io/projected/ba39174e-fd3a-4edc-8926-493fa7c4d650-kube-api-access-dkmq7\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.922807 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.922683    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.922807 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.922715    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.922807 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.922747    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.923235 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.923208    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.923235 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.923226    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.923331 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.923293    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.925013 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.924993    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba39174e-fd3a-4edc-8926-493fa7c4d650-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.925199 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.925178    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba39174e-fd3a-4edc-8926-493fa7c4d650-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:07.931599 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:07.931576    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkmq7\" (UniqueName: \"kubernetes.io/projected/ba39174e-fd3a-4edc-8926-493fa7c4d650-kube-api-access-dkmq7\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng\" (UID: \"ba39174e-fd3a-4edc-8926-493fa7c4d650\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:08.062057 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:08.062027    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:08.191234 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:08.191211    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"]
Jun 02 02:59:08.193862 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:59:08.193824    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba39174e_fd3a_4edc_8926_493fa7c4d650.slice/crio-62b5f5a2e24e11c980fb5ba3d17d44884a9a1f8df0a603e3780243bcf82ea95d WatchSource:0}: Error finding container 62b5f5a2e24e11c980fb5ba3d17d44884a9a1f8df0a603e3780243bcf82ea95d: Status 404 returned error can't find the container with id 62b5f5a2e24e11c980fb5ba3d17d44884a9a1f8df0a603e3780243bcf82ea95d
Jun 02 02:59:08.523892 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:08.523805    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng" event={"ID":"ba39174e-fd3a-4edc-8926-493fa7c4d650","Type":"ContainerStarted","Data":"f966be3d75480261e4df35f6eb4a3093e9e0c7c2dd7a2f5c4b976c43f585753e"}
Jun 02 02:59:08.523892 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:08.523849    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng" event={"ID":"ba39174e-fd3a-4edc-8926-493fa7c4d650","Type":"ContainerStarted","Data":"62b5f5a2e24e11c980fb5ba3d17d44884a9a1f8df0a603e3780243bcf82ea95d"}
Jun 02 02:59:09.528401 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:09.528368    2554 generic.go:358] "Generic (PLEG): container finished" podID="ba39174e-fd3a-4edc-8926-493fa7c4d650" containerID="f966be3d75480261e4df35f6eb4a3093e9e0c7c2dd7a2f5c4b976c43f585753e" exitCode=0
Jun 02 02:59:09.528835 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:09.528448    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng" event={"ID":"ba39174e-fd3a-4edc-8926-493fa7c4d650","Type":"ContainerDied","Data":"f966be3d75480261e4df35f6eb4a3093e9e0c7c2dd7a2f5c4b976c43f585753e"}
Jun 02 02:59:10.533389 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.533354    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng" event={"ID":"ba39174e-fd3a-4edc-8926-493fa7c4d650","Type":"ContainerStarted","Data":"e8dbfb04bbc7016f0391388a6eb81fa01b203ff3aa3e9d5836eacf8445f5510e"}
Jun 02 02:59:10.550284 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.550230    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng" podStartSLOduration=3.358445879 podStartE2EDuration="3.550211983s" podCreationTimestamp="2026-06-02 02:59:07 +0000 UTC" firstStartedPulling="2026-06-02 02:59:09.529104615 +0000 UTC m=+838.145593687" lastFinishedPulling="2026-06-02 02:59:09.720870718 +0000 UTC m=+838.337359791" observedRunningTime="2026-06-02 02:59:10.549817459 +0000 UTC m=+839.166306549" watchObservedRunningTime="2026-06-02 02:59:10.550211983 +0000 UTC m=+839.166701077"
Jun 02 02:59:10.640801 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.640762    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"]
Jun 02 02:59:10.644230 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.644206    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.646589 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.646571    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jun 02 02:59:10.652386 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.652356    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"]
Jun 02 02:59:10.755552 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.755519    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.755675 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.755560    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.755675 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.755612    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.755675 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.755672    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.755797 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.755702    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.755797 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.755777    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7gj7\" (UniqueName: \"kubernetes.io/projected/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-kube-api-access-j7gj7\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.856264 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.856229    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j7gj7\" (UniqueName: \"kubernetes.io/projected/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-kube-api-access-j7gj7\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.856439 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.856298    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.856439 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.856327    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.856439 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.856355    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.856439 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.856405    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.856650 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.856444    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.856813 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.856783    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.856880 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.856807    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.856880 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.856839    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.858700 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.858680    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.858808 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.858791    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.864088 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.864068    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7gj7\" (UniqueName: \"kubernetes.io/projected/3cc099c8-8feb-4b7d-8f86-3f3c588d6245-kube-api-access-j7gj7\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd\" (UID: \"3cc099c8-8feb-4b7d-8f86-3f3c588d6245\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:10.953909 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:10.953866    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:11.118100 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:11.118074    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"]
Jun 02 02:59:11.120084 ip-10-0-129-101 kubenswrapper[2554]: W0602 02:59:11.120053    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc099c8_8feb_4b7d_8f86_3f3c588d6245.slice/crio-1deb987a0078ee95d62025d0149303c020ef6b49dcdb5950e94dd3b2f2faf80b WatchSource:0}: Error finding container 1deb987a0078ee95d62025d0149303c020ef6b49dcdb5950e94dd3b2f2faf80b: Status 404 returned error can't find the container with id 1deb987a0078ee95d62025d0149303c020ef6b49dcdb5950e94dd3b2f2faf80b
Jun 02 02:59:11.537799 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:11.537720    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd" event={"ID":"3cc099c8-8feb-4b7d-8f86-3f3c588d6245","Type":"ContainerStarted","Data":"97eb2e396f1441302a07d668d4d877ea6c95007e1e2e717de69b00fe2756381a"}
Jun 02 02:59:11.537799 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:11.537760    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd" event={"ID":"3cc099c8-8feb-4b7d-8f86-3f3c588d6245","Type":"ContainerStarted","Data":"1deb987a0078ee95d62025d0149303c020ef6b49dcdb5950e94dd3b2f2faf80b"}
Jun 02 02:59:15.553615 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:15.553576    2554 generic.go:358] "Generic (PLEG): container finished" podID="3cc099c8-8feb-4b7d-8f86-3f3c588d6245" containerID="97eb2e396f1441302a07d668d4d877ea6c95007e1e2e717de69b00fe2756381a" exitCode=0
Jun 02 02:59:15.553952 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:15.553641    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd" event={"ID":"3cc099c8-8feb-4b7d-8f86-3f3c588d6245","Type":"ContainerDied","Data":"97eb2e396f1441302a07d668d4d877ea6c95007e1e2e717de69b00fe2756381a"}
Jun 02 02:59:16.558468 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:16.558430    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd" event={"ID":"3cc099c8-8feb-4b7d-8f86-3f3c588d6245","Type":"ContainerStarted","Data":"d9c51c99a6981004a9d42281647b6fcaeb2c436c1511cf41618e58173bdf5940"}
Jun 02 02:59:16.585995 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:16.585941    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd" podStartSLOduration=6.40332382 podStartE2EDuration="6.585925244s" podCreationTimestamp="2026-06-02 02:59:10 +0000 UTC" firstStartedPulling="2026-06-02 02:59:15.554261014 +0000 UTC m=+844.170750084" lastFinishedPulling="2026-06-02 02:59:15.736862425 +0000 UTC m=+844.353351508" observedRunningTime="2026-06-02 02:59:16.584650684 +0000 UTC m=+845.201139982" watchObservedRunningTime="2026-06-02 02:59:16.585925244 +0000 UTC m=+845.202414335"
Jun 02 02:59:18.062960 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:18.062927    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:18.062960 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:18.062970    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:18.075558 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:18.075536    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:18.580670 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:18.580642    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng"
Jun 02 02:59:20.954821 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:20.954790    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:20.955190 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:20.955070    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:20.967419 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:20.967398    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 02:59:21.589023 ip-10-0-129-101 kubenswrapper[2554]: I0602 02:59:21.588993    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd"
Jun 02 03:00:11.913853 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:11.913771    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 03:00:11.914486 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:11.914445    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 03:00:23.043064 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.043029    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-567597bf7-8ftkc"]
Jun 02 03:00:23.045166 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.045149    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.060560 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.060526    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-567597bf7-8ftkc"]
Jun 02 03:00:23.173772 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.173731    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r952q\" (UniqueName: \"kubernetes.io/projected/916c6c58-d8b2-43ae-8943-2d6b99bc77e7-kube-api-access-r952q\") pod \"authorino-567597bf7-8ftkc\" (UID: \"916c6c58-d8b2-43ae-8943-2d6b99bc77e7\") " pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.173772 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.173772    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/916c6c58-d8b2-43ae-8943-2d6b99bc77e7-tls-cert\") pod \"authorino-567597bf7-8ftkc\" (UID: \"916c6c58-d8b2-43ae-8943-2d6b99bc77e7\") " pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.174034 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.173988    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/916c6c58-d8b2-43ae-8943-2d6b99bc77e7-oidc-ca\") pod \"authorino-567597bf7-8ftkc\" (UID: \"916c6c58-d8b2-43ae-8943-2d6b99bc77e7\") " pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.275532 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.275487    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/916c6c58-d8b2-43ae-8943-2d6b99bc77e7-oidc-ca\") pod \"authorino-567597bf7-8ftkc\" (UID: \"916c6c58-d8b2-43ae-8943-2d6b99bc77e7\") " pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.275714 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.275594    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r952q\" (UniqueName: \"kubernetes.io/projected/916c6c58-d8b2-43ae-8943-2d6b99bc77e7-kube-api-access-r952q\") pod \"authorino-567597bf7-8ftkc\" (UID: \"916c6c58-d8b2-43ae-8943-2d6b99bc77e7\") " pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.275714 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.275615    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/916c6c58-d8b2-43ae-8943-2d6b99bc77e7-tls-cert\") pod \"authorino-567597bf7-8ftkc\" (UID: \"916c6c58-d8b2-43ae-8943-2d6b99bc77e7\") " pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.276119 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.276098    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/916c6c58-d8b2-43ae-8943-2d6b99bc77e7-oidc-ca\") pod \"authorino-567597bf7-8ftkc\" (UID: \"916c6c58-d8b2-43ae-8943-2d6b99bc77e7\") " pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.278142 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.278119    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/916c6c58-d8b2-43ae-8943-2d6b99bc77e7-tls-cert\") pod \"authorino-567597bf7-8ftkc\" (UID: \"916c6c58-d8b2-43ae-8943-2d6b99bc77e7\") " pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.282902 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.282880    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r952q\" (UniqueName: \"kubernetes.io/projected/916c6c58-d8b2-43ae-8943-2d6b99bc77e7-kube-api-access-r952q\") pod \"authorino-567597bf7-8ftkc\" (UID: \"916c6c58-d8b2-43ae-8943-2d6b99bc77e7\") " pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.360815 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.360771    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-567597bf7-8ftkc"
Jun 02 03:00:23.486527 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.486331    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-567597bf7-8ftkc"]
Jun 02 03:00:23.489529 ip-10-0-129-101 kubenswrapper[2554]: W0602 03:00:23.489488    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod916c6c58_d8b2_43ae_8943_2d6b99bc77e7.slice/crio-fefe5689c011030f6801001f4f3ba17ed829740c2366f6eadde1d96e8f1d834d WatchSource:0}: Error finding container fefe5689c011030f6801001f4f3ba17ed829740c2366f6eadde1d96e8f1d834d: Status 404 returned error can't find the container with id fefe5689c011030f6801001f4f3ba17ed829740c2366f6eadde1d96e8f1d834d
Jun 02 03:00:23.789153 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:23.789057    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-567597bf7-8ftkc" event={"ID":"916c6c58-d8b2-43ae-8943-2d6b99bc77e7","Type":"ContainerStarted","Data":"fefe5689c011030f6801001f4f3ba17ed829740c2366f6eadde1d96e8f1d834d"}
Jun 02 03:00:24.794589 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:24.794551    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-567597bf7-8ftkc" event={"ID":"916c6c58-d8b2-43ae-8943-2d6b99bc77e7","Type":"ContainerStarted","Data":"64378e3ba20ed0564a75e51e900f5872130424780973521aa8a93f90b90c3643"}
Jun 02 03:00:24.816415 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:24.816345    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-567597bf7-8ftkc" podStartSLOduration=1.237594255 podStartE2EDuration="1.816327123s" podCreationTimestamp="2026-06-02 03:00:23 +0000 UTC" firstStartedPulling="2026-06-02 03:00:23.491136842 +0000 UTC m=+912.107625910" lastFinishedPulling="2026-06-02 03:00:24.069869703 +0000 UTC m=+912.686358778" observedRunningTime="2026-06-02 03:00:24.814902391 +0000 UTC m=+913.431391485" watchObservedRunningTime="2026-06-02 03:00:24.816327123 +0000 UTC m=+913.432816217"
Jun 02 03:00:24.843544 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:24.843501    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78f58594b8-2888q"]
Jun 02 03:00:24.843868 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:24.843837    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-78f58594b8-2888q" podUID="a8543526-fa97-4d98-a117-c8407c426358" containerName="authorino" containerID="cri-o://8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77" gracePeriod=30
Jun 02 03:00:25.114116 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.114092    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 03:00:25.190812 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.190782    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a8543526-fa97-4d98-a117-c8407c426358-oidc-ca\") pod \"a8543526-fa97-4d98-a117-c8407c426358\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") "
Jun 02 03:00:25.190980 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.190839    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sxgz\" (UniqueName: \"kubernetes.io/projected/a8543526-fa97-4d98-a117-c8407c426358-kube-api-access-6sxgz\") pod \"a8543526-fa97-4d98-a117-c8407c426358\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") "
Jun 02 03:00:25.190980 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.190917    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8543526-fa97-4d98-a117-c8407c426358-tls-cert\") pod \"a8543526-fa97-4d98-a117-c8407c426358\" (UID: \"a8543526-fa97-4d98-a117-c8407c426358\") "
Jun 02 03:00:25.192936 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.192897    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8543526-fa97-4d98-a117-c8407c426358-kube-api-access-6sxgz" (OuterVolumeSpecName: "kube-api-access-6sxgz") pod "a8543526-fa97-4d98-a117-c8407c426358" (UID: "a8543526-fa97-4d98-a117-c8407c426358"). InnerVolumeSpecName "kube-api-access-6sxgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 03:00:25.195374 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.195344    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8543526-fa97-4d98-a117-c8407c426358-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "a8543526-fa97-4d98-a117-c8407c426358" (UID: "a8543526-fa97-4d98-a117-c8407c426358"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 03:00:25.201653 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.201624    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8543526-fa97-4d98-a117-c8407c426358-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a8543526-fa97-4d98-a117-c8407c426358" (UID: "a8543526-fa97-4d98-a117-c8407c426358"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 03:00:25.291779 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.291747    2554 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8543526-fa97-4d98-a117-c8407c426358-tls-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 03:00:25.291779 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.291776    2554 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a8543526-fa97-4d98-a117-c8407c426358-oidc-ca\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 03:00:25.291942 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.291789    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sxgz\" (UniqueName: \"kubernetes.io/projected/a8543526-fa97-4d98-a117-c8407c426358-kube-api-access-6sxgz\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 03:00:25.798272 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.798237    2554 generic.go:358] "Generic (PLEG): container finished" podID="a8543526-fa97-4d98-a117-c8407c426358" containerID="8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77" exitCode=0
Jun 02 03:00:25.798719 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.798286    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f58594b8-2888q"
Jun 02 03:00:25.798719 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.798328    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f58594b8-2888q" event={"ID":"a8543526-fa97-4d98-a117-c8407c426358","Type":"ContainerDied","Data":"8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77"}
Jun 02 03:00:25.798719 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.798367    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f58594b8-2888q" event={"ID":"a8543526-fa97-4d98-a117-c8407c426358","Type":"ContainerDied","Data":"d5705abba0bd47ca2cc98622799b437eef812535f949db06fe5e33d095133e6e"}
Jun 02 03:00:25.798719 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.798385    2554 scope.go:117] "RemoveContainer" containerID="8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77"
Jun 02 03:00:25.806390 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.806370    2554 scope.go:117] "RemoveContainer" containerID="8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77"
Jun 02 03:00:25.806671 ip-10-0-129-101 kubenswrapper[2554]: E0602 03:00:25.806649    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77\": container with ID starting with 8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77 not found: ID does not exist" containerID="8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77"
Jun 02 03:00:25.806724 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.806684    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77"} err="failed to get container status \"8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77\": rpc error: code = NotFound desc = could not find container \"8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77\": container with ID starting with 8659feecbe4f234339ff4818d8ab2d16808ac41fd2326323f24c96b8c916ed77 not found: ID does not exist"
Jun 02 03:00:25.818340 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.818319    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78f58594b8-2888q"]
Jun 02 03:00:25.827694 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:25.827670    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-78f58594b8-2888q"]
Jun 02 03:00:26.000428 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:00:26.000392    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8543526-fa97-4d98-a117-c8407c426358" path="/var/lib/kubelet/pods/a8543526-fa97-4d98-a117-c8407c426358/volumes"
Jun 02 03:01:51.585000 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:51.584956    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86dc8749c7-8hrtr"]
Jun 02 03:01:51.585563 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:51.585252    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-86dc8749c7-8hrtr" podUID="04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc" containerName="manager" containerID="cri-o://1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa" gracePeriod=30
Jun 02 03:01:51.830628 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:51.830604    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc8749c7-8hrtr"
Jun 02 03:01:51.929489 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:51.929374    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzqnw\" (UniqueName: \"kubernetes.io/projected/04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc-kube-api-access-pzqnw\") pod \"04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc\" (UID: \"04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc\") "
Jun 02 03:01:51.931474 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:51.931429    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc-kube-api-access-pzqnw" (OuterVolumeSpecName: "kube-api-access-pzqnw") pod "04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc" (UID: "04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc"). InnerVolumeSpecName "kube-api-access-pzqnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 03:01:52.030380 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.030351    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzqnw\" (UniqueName: \"kubernetes.io/projected/04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc-kube-api-access-pzqnw\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\""
Jun 02 03:01:52.083061 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.083028    2554 generic.go:358] "Generic (PLEG): container finished" podID="04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc" containerID="1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa" exitCode=0
Jun 02 03:01:52.083174 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.083096    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc8749c7-8hrtr"
Jun 02 03:01:52.083174 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.083096    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc8749c7-8hrtr" event={"ID":"04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc","Type":"ContainerDied","Data":"1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa"}
Jun 02 03:01:52.083268 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.083194    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc8749c7-8hrtr" event={"ID":"04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc","Type":"ContainerDied","Data":"cff1ca3021a5b4f5746d9bfab5da22368c3c1c1bde5e19eb53c62f6fee5c248e"}
Jun 02 03:01:52.083268 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.083213    2554 scope.go:117] "RemoveContainer" containerID="1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa"
Jun 02 03:01:52.092864 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.092836    2554 scope.go:117] "RemoveContainer" containerID="1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa"
Jun 02 03:01:52.093108 ip-10-0-129-101 kubenswrapper[2554]: E0602 03:01:52.093086    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa\": container with ID starting with 1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa not found: ID does not exist" containerID="1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa"
Jun 02 03:01:52.093161 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.093117    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa"} err="failed to get container status \"1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa\": rpc error: code = NotFound desc = could not find container \"1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa\": container with ID starting with 1789fb199810d52d8f8089b99a94b9f668d410e1e5cd965a9bb931b6bf0063aa not found: ID does not exist"
Jun 02 03:01:52.100780 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.100757    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86dc8749c7-8hrtr"]
Jun 02 03:01:52.104533 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.104508    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86dc8749c7-8hrtr"]
Jun 02 03:01:52.823764 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.823732    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86dc8749c7-vm5hd"]
Jun 02 03:01:52.824151 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.824039    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8543526-fa97-4d98-a117-c8407c426358" containerName="authorino"
Jun 02 03:01:52.824151 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.824051    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8543526-fa97-4d98-a117-c8407c426358" containerName="authorino"
Jun 02 03:01:52.824151 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.824069    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc" containerName="manager"
Jun 02 03:01:52.824151 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.824074    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc" containerName="manager"
Jun 02 03:01:52.824151 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.824124    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="a8543526-fa97-4d98-a117-c8407c426358" containerName="authorino"
Jun 02 03:01:52.824151 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.824135    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc" containerName="manager"
Jun 02 03:01:52.826735 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.826719    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc8749c7-vm5hd"
Jun 02 03:01:52.829200 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.829174    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jun 02 03:01:52.829336 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.829261    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kqxff\""
Jun 02 03:01:52.836232 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.836209    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86dc8749c7-vm5hd"]
Jun 02 03:01:52.938625 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:52.938595    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25jj5\" (UniqueName: \"kubernetes.io/projected/6011b879-51c9-4803-bfab-396ec06b1a98-kube-api-access-25jj5\") pod \"maas-controller-86dc8749c7-vm5hd\" (UID: \"6011b879-51c9-4803-bfab-396ec06b1a98\") " pod="opendatahub/maas-controller-86dc8749c7-vm5hd"
Jun 02 03:01:53.039135 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:53.039098    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-25jj5\" (UniqueName: \"kubernetes.io/projected/6011b879-51c9-4803-bfab-396ec06b1a98-kube-api-access-25jj5\") pod \"maas-controller-86dc8749c7-vm5hd\" (UID: \"6011b879-51c9-4803-bfab-396ec06b1a98\") " pod="opendatahub/maas-controller-86dc8749c7-vm5hd"
Jun 02 03:01:53.046899 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:53.046877    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25jj5\" (UniqueName: \"kubernetes.io/projected/6011b879-51c9-4803-bfab-396ec06b1a98-kube-api-access-25jj5\") pod \"maas-controller-86dc8749c7-vm5hd\" (UID: \"6011b879-51c9-4803-bfab-396ec06b1a98\") " pod="opendatahub/maas-controller-86dc8749c7-vm5hd"
Jun 02 03:01:53.137201 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:53.137112    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc8749c7-vm5hd"
Jun 02 03:01:53.262334 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:53.262310    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86dc8749c7-vm5hd"]
Jun 02 03:01:53.265343 ip-10-0-129-101 kubenswrapper[2554]: W0602 03:01:53.265314    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6011b879_51c9_4803_bfab_396ec06b1a98.slice/crio-a783c01efb1b0cfad30a5421626bd225254ee72e786266cad44a68792e978a17 WatchSource:0}: Error finding container a783c01efb1b0cfad30a5421626bd225254ee72e786266cad44a68792e978a17: Status 404 returned error can't find the container with id a783c01efb1b0cfad30a5421626bd225254ee72e786266cad44a68792e978a17
Jun 02 03:01:54.000635 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:54.000549    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc" path="/var/lib/kubelet/pods/04036ab1-cbdc-40f4-a3e2-f4d87ccd90dc/volumes"
Jun 02 03:01:54.090873 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:54.090836    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc8749c7-vm5hd" event={"ID":"6011b879-51c9-4803-bfab-396ec06b1a98","Type":"ContainerStarted","Data":"6265cacdeae38eb3ddb6c8eb2033a2e7ec5239f671163816b2a1014afc95725c"}
Jun 02 03:01:54.090873 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:54.090875    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc8749c7-vm5hd" event={"ID":"6011b879-51c9-4803-bfab-396ec06b1a98","Type":"ContainerStarted","Data":"a783c01efb1b0cfad30a5421626bd225254ee72e786266cad44a68792e978a17"}
Jun 02 03:01:54.091077 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:54.090931    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86dc8749c7-vm5hd"
Jun 02 03:01:54.105944 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:01:54.105894    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-86dc8749c7-vm5hd" podStartSLOduration=1.7302818599999998 podStartE2EDuration="2.105877231s" podCreationTimestamp="2026-06-02 03:01:52 +0000 UTC" firstStartedPulling="2026-06-02 03:01:53.266700927 +0000 UTC m=+1001.883190000" lastFinishedPulling="2026-06-02 03:01:53.642296297 +0000 UTC m=+1002.258785371" observedRunningTime="2026-06-02 03:01:54.10493325 +0000 UTC m=+1002.721422350" watchObservedRunningTime="2026-06-02 03:01:54.105877231 +0000 UTC m=+1002.722366321"
Jun 02 03:02:05.099501 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:02:05.099442    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86dc8749c7-vm5hd"
Jun 02 03:04:51.831987 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:04:51.831948    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-567597bf7-8ftkc_916c6c58-d8b2-43ae-8943-2d6b99bc77e7/authorino/0.log"
Jun 02 03:04:56.987553 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:04:56.987522    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5d9b6489bc-hm652_3bda536c-adc8-4fbc-87d7-3885775be5d2/maas-api/0.log"
Jun 02 03:04:57.220154 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:04:57.220119    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86dc8749c7-vm5hd_6011b879-51c9-4803-bfab-396ec06b1a98/manager/0.log"
Jun 02 03:04:57.471845 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:04:57.471809    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-2qq4b_0e25396f-883d-4114-8451-8359c3ecf4f0/manager/0.log"
Jun 02 03:04:57.725300 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:04:57.725225    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f8cc9fd78-vnlqd_521bc591-2a8d-4436-92f3-ec2923cdad98/copy-manifests/0.log"
Jun 02 03:04:57.732812 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:04:57.732793    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f8cc9fd78-vnlqd_521bc591-2a8d-4436-92f3-ec2923cdad98/manager/0.log"
Jun 02 03:04:59.298312 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:04:59.298278    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-567597bf7-8ftkc_916c6c58-d8b2-43ae-8943-2d6b99bc77e7/authorino/0.log"
Jun 02 03:04:59.677807 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:04:59.677729    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-mzh86_f6445b01-c28d-4fa9-af7d-7fcce384f1a4/kuadrant-console-plugin/0.log"
Jun 02 03:05:00.746339 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:00.746306    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85d86d5b57-2264c_99546707-eb92-4ba6-a09b-e33b2186434c/kube-auth-proxy/0.log"
Jun 02 03:05:01.739184 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:01.739155    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-ts677_037ccd98-3a71-411b-a07c-09d47c42743f/storage-initializer/0.log"
Jun 02 03:05:01.747701 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:01.747674    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-ts677_037ccd98-3a71-411b-a07c-09d47c42743f/main/0.log"
Jun 02 03:05:01.992770 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:01.992695    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm_b22eca10-be94-4010-88ee-fc1e9b0ced87/storage-initializer/0.log"
Jun 02 03:05:02.000877 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:02.000856    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f97pxwm_b22eca10-be94-4010-88ee-fc1e9b0ced87/main/0.log"
Jun 02 03:05:02.124875 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:02.124841    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng_ba39174e-fd3a-4edc-8926-493fa7c4d650/storage-initializer/0.log"
Jun 02 03:05:02.133607 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:02.133572    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-74qng_ba39174e-fd3a-4edc-8926-493fa7c4d650/main/0.log"
Jun 02 03:05:02.246046 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:02.245972    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd_3cc099c8-8feb-4b7d-8f86-3f3c588d6245/main/0.log"
Jun 02 03:05:02.252818 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:02.252789    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-drcpd_3cc099c8-8feb-4b7d-8f86-3f3c588d6245/storage-initializer/0.log"
Jun 02 03:05:09.721126 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:09.721093    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fln9v_7093c19d-4a2c-4d50-aada-46c00f5c5dd7/global-pull-secret-syncer/0.log"
Jun 02 03:05:09.960999 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:09.960966    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-msg6m_adc9d363-18a3-4d49-a07c-e13c52572722/konnectivity-agent/0.log"
Jun 02 03:05:10.001513 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:10.001408    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-101.ec2.internal_ccd85c0378921da30d2712ae09333302/haproxy/0.log"
Jun 02 03:05:11.935537 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:11.935511    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 03:05:11.937409 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:11.937388    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ws7xp_59e50745-ea12-4b4d-9707-0776345bfe9c/ovn-acl-logging/0.log"
Jun 02 03:05:13.875395 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:13.875361    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-567597bf7-8ftkc_916c6c58-d8b2-43ae-8943-2d6b99bc77e7/authorino/0.log"
Jun 02 03:05:14.007274 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:14.007246    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-mzh86_f6445b01-c28d-4fa9-af7d-7fcce384f1a4/kuadrant-console-plugin/0.log"
Jun 02 03:05:16.553792 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:16.553754    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8lrv9_d4af44da-ca2e-4e34-8828-0a2791b8d274/node-exporter/0.log"
Jun 02 03:05:16.584012 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:16.583989    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8lrv9_d4af44da-ca2e-4e34-8828-0a2791b8d274/kube-rbac-proxy/0.log"
Jun 02 03:05:16.611303 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:16.611280    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8lrv9_d4af44da-ca2e-4e34-8828-0a2791b8d274/init-textfile/0.log"
Jun 02 03:05:18.685274 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.685243    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-m6jzt_e3d7f74d-20a3-484c-b9ca-4ab73b6a292c/networking-console-plugin/0.log"
Jun 02 03:05:18.774564 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.774530    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"]
Jun 02 03:05:18.778005 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.777984    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.780643 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.780622    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sfk5k\"/\"kube-root-ca.crt\""
Jun 02 03:05:18.781648 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.781626    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sfk5k\"/\"openshift-service-ca.crt\""
Jun 02 03:05:18.781758 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.781626    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sfk5k\"/\"default-dockercfg-8xdm6\""
Jun 02 03:05:18.787359 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.787339    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"]
Jun 02 03:05:18.872375 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.872344    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-proc\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.872588 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.872383    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-podres\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.872588 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.872569    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-lib-modules\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.872714 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.872608    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsp4m\" (UniqueName: \"kubernetes.io/projected/94d5e3c4-0166-45ec-8458-eb03b66c4f32-kube-api-access-qsp4m\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.872714 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.872671    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-sys\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.973752 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.973671    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-lib-modules\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.973752 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.973708    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qsp4m\" (UniqueName: \"kubernetes.io/projected/94d5e3c4-0166-45ec-8458-eb03b66c4f32-kube-api-access-qsp4m\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.973752 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.973734    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-sys\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.974000 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.973755    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-proc\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.974000 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.973791    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-podres\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.974000 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.973829    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-sys\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.974000 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.973840    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-lib-modules\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.974000 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.973890    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-proc\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.974000 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.973893    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94d5e3c4-0166-45ec-8458-eb03b66c4f32-podres\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:18.984767 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:18.984745    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsp4m\" (UniqueName: \"kubernetes.io/projected/94d5e3c4-0166-45ec-8458-eb03b66c4f32-kube-api-access-qsp4m\") pod \"perf-node-gather-daemonset-hddzk\" (UID: \"94d5e3c4-0166-45ec-8458-eb03b66c4f32\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:19.089176 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:19.089151    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:19.224781 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:19.224730    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"]
Jun 02 03:05:19.225802 ip-10-0-129-101 kubenswrapper[2554]: W0602 03:05:19.225774    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod94d5e3c4_0166_45ec_8458_eb03b66c4f32.slice/crio-6f48962c456b13baaea7750043756bf4cf2243d0e51975b77e92ba6854f04b93 WatchSource:0}: Error finding container 6f48962c456b13baaea7750043756bf4cf2243d0e51975b77e92ba6854f04b93: Status 404 returned error can't find the container with id 6f48962c456b13baaea7750043756bf4cf2243d0e51975b77e92ba6854f04b93
Jun 02 03:05:19.227665 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:19.227641    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 03:05:19.750690 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:19.750650    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk" event={"ID":"94d5e3c4-0166-45ec-8458-eb03b66c4f32","Type":"ContainerStarted","Data":"cce89b4c37c0af1c52ce02cca2581b4819e81412ab40617533bfd6ad4d5281fd"}
Jun 02 03:05:19.750690 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:19.750693    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk" event={"ID":"94d5e3c4-0166-45ec-8458-eb03b66c4f32","Type":"ContainerStarted","Data":"6f48962c456b13baaea7750043756bf4cf2243d0e51975b77e92ba6854f04b93"}
Jun 02 03:05:19.751122 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:19.750716    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:19.778393 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:19.778353    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk" podStartSLOduration=1.77833665 podStartE2EDuration="1.77833665s" podCreationTimestamp="2026-06-02 03:05:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 03:05:19.777516318 +0000 UTC m=+1208.394005420" watchObservedRunningTime="2026-06-02 03:05:19.77833665 +0000 UTC m=+1208.394825742"
Jun 02 03:05:21.629579 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:21.629549    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9x5hx_db50fd2b-9b3f-403f-a131-bf4a898ca513/dns/0.log"
Jun 02 03:05:21.653748 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:21.653724    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9x5hx_db50fd2b-9b3f-403f-a131-bf4a898ca513/kube-rbac-proxy/0.log"
Jun 02 03:05:21.680637 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:21.680609    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cn8tf_b170f48b-6b05-4361-90e3-4f5b81d273ea/dns-node-resolver/0.log"
Jun 02 03:05:22.241844 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:22.241819    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4x78s_23a084fd-4b0c-4c56-930d-b946a7e1bcbe/node-ca/0.log"
Jun 02 03:05:23.241773 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:23.241749    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85d86d5b57-2264c_99546707-eb92-4ba6-a09b-e33b2186434c/kube-auth-proxy/0.log"
Jun 02 03:05:23.951933 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:23.951904    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nhglj_a457d44f-df4a-4ef5-9de4-632e3318feee/serve-healthcheck-canary/0.log"
Jun 02 03:05:24.477647 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:24.477621    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-knmcm_84bdde3a-4a50-45d9-b0ea-ea65680d0a06/kube-rbac-proxy/0.log"
Jun 02 03:05:24.500299 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:24.500274    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-knmcm_84bdde3a-4a50-45d9-b0ea-ea65680d0a06/exporter/0.log"
Jun 02 03:05:24.523930 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:24.523906    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-knmcm_84bdde3a-4a50-45d9-b0ea-ea65680d0a06/extractor/0.log"
Jun 02 03:05:25.763669 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:25.763638    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-hddzk"
Jun 02 03:05:26.866050 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:26.866019    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5d9b6489bc-hm652_3bda536c-adc8-4fbc-87d7-3885775be5d2/maas-api/0.log"
Jun 02 03:05:26.933295 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:26.933267    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86dc8749c7-vm5hd_6011b879-51c9-4803-bfab-396ec06b1a98/manager/0.log"
Jun 02 03:05:27.034831 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:27.034806    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-2qq4b_0e25396f-883d-4114-8451-8359c3ecf4f0/manager/0.log"
Jun 02 03:05:27.204096 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:27.204067    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f8cc9fd78-vnlqd_521bc591-2a8d-4436-92f3-ec2923cdad98/manager/0.log"
Jun 02 03:05:27.259781 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:27.259755    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f8cc9fd78-vnlqd_521bc591-2a8d-4436-92f3-ec2923cdad98/copy-manifests/0.log"
Jun 02 03:05:29.115862 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:29.115831    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-774f9bf4f6-fplp6_93401a05-a109-4fb2-a703-5d8e4a3f3ba9/manager/0.log"
Jun 02 03:05:35.671768 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:35.671739    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4stzd_007548f0-391f-4fc2-8977-615a8a04f403/kube-multus-additional-cni-plugins/0.log"
Jun 02 03:05:35.693302 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:35.693265    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4stzd_007548f0-391f-4fc2-8977-615a8a04f403/egress-router-binary-copy/0.log"
Jun 02 03:05:35.714293 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:35.714268    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4stzd_007548f0-391f-4fc2-8977-615a8a04f403/cni-plugins/0.log"
Jun 02 03:05:35.738444 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:35.738419    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4stzd_007548f0-391f-4fc2-8977-615a8a04f403/bond-cni-plugin/0.log"
Jun 02 03:05:35.760472 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:35.760432    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4stzd_007548f0-391f-4fc2-8977-615a8a04f403/routeoverride-cni/0.log"
Jun 02 03:05:35.780737 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:35.780709    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4stzd_007548f0-391f-4fc2-8977-615a8a04f403/whereabouts-cni-bincopy/0.log"
Jun 02 03:05:35.801584 ip-10-0-129-101 kubenswrapper[2554]: I0602 03:05:35.801553    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4stzd_007548f0-391f-4fc2-8977-615a8a04f403/whereabouts-cni/0.log"