Jun 10 15:01:36.742167 ip-10-0-129-85 systemd[1]: Starting Kubernetes Kubelet... Jun 10 15:01:37.181032 ip-10-0-129-85 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 15:01:37.181032 ip-10-0-129-85 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 10 15:01:37.181032 ip-10-0-129-85 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 15:01:37.181032 ip-10-0-129-85 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 10 15:01:37.181032 ip-10-0-129-85 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 15:01:37.184624 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.184559 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 10 15:01:37.189922 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189908 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 15:01:37.189922 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189922 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189937 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189941 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189944 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189947 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189951 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189953 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189956 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189959 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189962 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189965 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189968 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189971 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189973 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189976 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189989 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189992 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189995 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.189997 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190000 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 15:01:37.190005 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190002 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190005 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190008 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190011 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190013 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190016 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190019 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190022 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190024 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190028 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190032 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190035 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190038 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190041 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190043 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190046 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190048 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190051 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190054 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 15:01:37.190549 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190056 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190058 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190061 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190063 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190066 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190069 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190071 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190074 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190077 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190080 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190082 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190085 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190087 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190090 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190093 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190096 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190099 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190101 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190104 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190107 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 15:01:37.191267 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190110 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190112 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190115 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190117 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190119 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190122 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190126 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190129 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190132 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190135 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190137 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190140 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190143 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190145 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190149 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190151 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190154 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190156 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190159 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190162 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 15:01:37.191864 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190164 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190167 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190169 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190172 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190174 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190177 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190180 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190183 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190185 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190188 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190191 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190193 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190196 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190199 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190201 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190204 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190208 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190211 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 15:01:37.192373 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.190214 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192383 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192393 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192397 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192400 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192403 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192406 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192409 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192411 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192414 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192417 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192419 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192422 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192424 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192427 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192430 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192433 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192435 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192438 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192440 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 15:01:37.192799 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192443 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192445 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192448 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192450 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192453 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192455 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192459 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192463 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192466 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192469 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192472 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192474 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192478 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192487 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192491 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192494 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192497 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192500 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192502 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 15:01:37.193346 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192505 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192508 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192511 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192514 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192516 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192518 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192521 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192523 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192526 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192528 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192531 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192534 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192537 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192539 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192542 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192544 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192547 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192549 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192552 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192554 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 15:01:37.193807 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192557 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192559 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192562 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192564 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192567 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192570 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192574 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192577 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192580 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192583 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192587 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192590 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192593 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192595 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192598 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192601 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192603 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192606 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192608 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192611 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192613 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 15:01:37.194322 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192616 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192623 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192626 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192628 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192631 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192634 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192637 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192639 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192642 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192644 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192646 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192649 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192652 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192654 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192657 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192659 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192662 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192665 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192668 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.192671 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 15:01:37.194833 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193030 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193036 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193039 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193042 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193044 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193047 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193050 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193052 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193055 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193058 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193061 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193064 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193066 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193069 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193071 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193074 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193077 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193080 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193083 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193086 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 15:01:37.195343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193088 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193091 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193093 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193096 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193098 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193101 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193104 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193106 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193110 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193112 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193117 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193121 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193124 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193127 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193130 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193132 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193135 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193143 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193146 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 15:01:37.195840 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193148 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193151 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193153 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193156 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193158 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193161 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193164 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193166 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193168 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193171 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193174 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193177 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193181 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193184 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193187 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193190 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193193 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193195 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193198 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 15:01:37.196320 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193200 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193203 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193206 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193208 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193212 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193214 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193217 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193220 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193223 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193226 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193229 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193232 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193234 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193237 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193240 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193242 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193244 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193247 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193249 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193252 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 15:01:37.196777 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193255 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193257 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193259 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193262 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193265 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193268 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193270 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193273 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193275 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193278 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193281 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193284 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193287 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193289 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193292 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193294 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193298 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193300 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193303 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 15:01:37.197271 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193305 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.193308 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193383 2561 flags.go:64] FLAG: --address="0.0.0.0" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193394 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193402 2561 flags.go:64] FLAG: --anonymous-auth="true" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193419 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193423 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193426 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193430 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193434 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193437 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193439 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193443 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193446 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193449 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193452 2561 flags.go:64] FLAG: --cgroup-root="" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193455 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193458 2561 flags.go:64] FLAG: --client-ca-file="" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193462 2561 flags.go:64] FLAG: --cloud-provider="external" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193465 2561 flags.go:64] FLAG: --cluster-dns="[]" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193470 2561 flags.go:64] FLAG: --cluster-domain="" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193473 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193476 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193479 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 10 15:01:37.197728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193483 2561 flags.go:64] FLAG: --container-log-max-files="5" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193486 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193489 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193492 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193495 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193499 2561 flags.go:64] FLAG: --contention-profiling="false" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193502 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193505 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193507 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193510 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193514 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193517 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193520 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193523 2561 flags.go:64] FLAG: --enable-load-reader="false" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193526 2561 flags.go:64] FLAG: --enable-server="true" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193528 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193532 2561 flags.go:64] FLAG: --event-burst="100" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193535 2561 flags.go:64] FLAG: --event-qps="50" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193538 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193541 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193544 2561 flags.go:64] FLAG: --eviction-hard="" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193547 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193550 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193554 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193557 2561 flags.go:64] FLAG: --eviction-soft="" Jun 10 15:01:37.198367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193559 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193562 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193566 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193569 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193572 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193574 2561 flags.go:64] FLAG: --fail-swap-on="true" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193577 2561 flags.go:64] FLAG: --feature-gates="" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193580 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193583 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193587 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193590 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193593 2561 flags.go:64] FLAG: --healthz-port="10248" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193596 2561 flags.go:64] FLAG: --help="false" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193600 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193603 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193606 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193608 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193612 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193615 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193618 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193620 2561 flags.go:64] FLAG: --image-service-endpoint="" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193623 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193626 2561 flags.go:64] FLAG: --kube-api-burst="100" Jun 10 15:01:37.199041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193628 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193631 2561 flags.go:64] FLAG: --kube-api-qps="50" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193634 2561 flags.go:64] FLAG: --kube-reserved="" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193637 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193639 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193642 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193645 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193648 2561 flags.go:64] FLAG: --lock-file="" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193650 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193653 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193656 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193661 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193664 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193667 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193670 2561 flags.go:64] FLAG: --logging-format="text" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193672 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193676 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193678 2561 flags.go:64] FLAG: --manifest-url="" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193681 2561 flags.go:64] FLAG: --manifest-url-header="" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193690 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193693 2561 flags.go:64] FLAG: --max-open-files="1000000" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193697 2561 flags.go:64] FLAG: --max-pods="110" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193700 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193703 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193706 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jun 10 15:01:37.199619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193709 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193712 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193715 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193718 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193725 2561 flags.go:64] FLAG: --node-status-max-images="50" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193728 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193731 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193734 2561 flags.go:64] FLAG: --pod-cidr="" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193736 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193742 2561 flags.go:64] FLAG: --pod-manifest-path="" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193744 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193748 2561 flags.go:64] FLAG: --pods-per-core="0" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193751 2561 flags.go:64] FLAG: --port="10250" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193753 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193756 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03b0198bc33d38e0c" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193760 2561 flags.go:64] FLAG: --qos-reserved="" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193762 2561 flags.go:64] FLAG: --read-only-port="10255" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193765 2561 flags.go:64] FLAG: --register-node="true" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193768 2561 flags.go:64] FLAG: --register-with-taints="" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193771 2561 flags.go:64] FLAG: --registry-burst="10" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193775 2561 flags.go:64] FLAG: --registry-qps="5" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193778 2561 flags.go:64] FLAG: --reserved-cpus="" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193780 2561 flags.go:64] FLAG: --reserved-memory="" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193784 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193787 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 10 15:01:37.200227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193790 2561 flags.go:64] FLAG: --rotate-certificates="false" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193793 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193795 2561 flags.go:64] FLAG: --runonce="false" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193798 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193801 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193804 2561 flags.go:64] FLAG: --seccomp-default="false" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193807 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193809 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193813 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193816 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193819 2561 flags.go:64] FLAG: --storage-driver-password="root" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193821 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193824 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193827 2561 flags.go:64] FLAG: --storage-driver-user="root" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193830 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193833 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193835 2561 flags.go:64] FLAG: --system-cgroups="" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193838 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193843 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193846 2561 flags.go:64] FLAG: --tls-cert-file="" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193849 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193854 2561 flags.go:64] FLAG: --tls-min-version="" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193857 2561 flags.go:64] FLAG: --tls-private-key-file="" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193859 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193862 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 10 15:01:37.200861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193865 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193868 2561 flags.go:64] FLAG: --v="2" Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193872 2561 flags.go:64] FLAG: --version="false" Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193876 2561 flags.go:64] FLAG: --vmodule="" Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193880 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.193883 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194017 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194022 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194025 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194028 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194031 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194033 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194036 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194038 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194041 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194044 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194046 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194049 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194051 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194053 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194056 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 15:01:37.201505 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194058 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194061 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194063 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194066 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194068 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194071 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194073 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194076 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194078 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194080 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194083 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194091 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194094 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194097 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194100 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194103 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194106 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194109 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194112 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 15:01:37.202041 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194114 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194118 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194121 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194123 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194126 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194129 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194131 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194134 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194136 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194139 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194141 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194144 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194146 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194149 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194152 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194154 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194157 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194159 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194162 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194164 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 15:01:37.202508 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194167 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194170 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194173 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194175 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194179 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194181 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194184 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194187 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194189 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194194 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194197 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194200 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194203 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194207 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194210 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194213 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194215 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194218 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194220 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 15:01:37.203018 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194224 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194227 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194230 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194233 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194236 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194239 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194242 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194245 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194248 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194250 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194253 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194256 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194259 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194261 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194264 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194266 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194269 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194272 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194275 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194278 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 15:01:37.203481 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194280 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194283 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194285 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194289 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194291 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.194294 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.194889 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.201216 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.201228 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.201246 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.201252 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201304 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201309 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201312 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201315 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 15:01:37.203965 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201319 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201322 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201325 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201328 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201330 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201333 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201336 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201339 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201341 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201344 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201347 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201349 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201352 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201355 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201357 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201360 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201362 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201365 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201367 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201370 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 15:01:37.204342 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201373 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201375 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201380 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201383 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201386 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201390 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201393 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201397 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201400 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201403 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201405 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201408 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201410 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201413 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201416 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201418 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201421 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201424 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201426 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 15:01:37.204812 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201429 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201432 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201434 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201437 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201439 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201442 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201445 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201448 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201450 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201453 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201455 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201458 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201460 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201463 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201466 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201469 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201471 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201474 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201478 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201481 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 15:01:37.205292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201484 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201487 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201489 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201492 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201494 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201498 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201502 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201505 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201507 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201510 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201513 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201516 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201518 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201522 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201526 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201528 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201531 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201534 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201537 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 15:01:37.205775 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201540 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201543 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201546 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201549 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201551 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201553 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201556 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201559 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201561 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201564 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201567 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201570 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201572 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201576 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201578 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201581 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 15:01:37.206263 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201584 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.201589 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201692 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201696 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201699 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201702 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201705 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201708 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201712 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201716 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201719 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201722 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201725 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201727 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 15:01:37.206648 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201730 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201733 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201735 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201738 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201740 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201743 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201746 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201748 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201751 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201753 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201756 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201758 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201761 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201764 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201767 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201769 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201773 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201776 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201779 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 15:01:37.207106 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201782 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201785 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201787 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201790 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201793 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201795 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201798 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201800 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201803 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201807 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201810 2561 feature_gate.go:328] unrecognized feature gate: Example Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201812 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201815 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201818 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201821 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201824 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201826 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201844 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201848 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 15:01:37.207581 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201850 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201853 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201856 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201859 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201862 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201864 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201867 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201870 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201873 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201875 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201878 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201882 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201884 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201887 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201890 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201892 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201895 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201897 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201900 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201902 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 15:01:37.208044 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201905 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201907 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201910 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201913 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201915 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201918 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201920 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201923 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201939 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201942 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201945 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201947 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201950 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201952 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201955 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201958 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201960 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201963 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201966 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201968 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 15:01:37.208507 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201971 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201974 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201976 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201979 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201982 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201984 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201987 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201989 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:37.201991 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.201996 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.202083 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.204210 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.205643 2561 server.go:1019] "Starting client certificate rotation" Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.205743 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 15:01:37.208992 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.205797 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 15:01:37.209604 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.209592 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 15:01:37.210872 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.210855 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 15:01:37.223400 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.223379 2561 log.go:25] "Validated CRI v1 runtime API" Jun 10 15:01:37.226138 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.226124 2561 log.go:25] "Validated CRI v1 image API" Jun 10 15:01:37.226889 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.226869 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 10 15:01:37.229428 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.229412 2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 adb3b044-b591-43c5-bab8-082e3432e702:/dev/nvme0n1p3 bb84f5a1-32ba-4c9a-84c2-f502136c67c1:/dev/nvme0n1p4] Jun 10 15:01:37.229494 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.229428 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 10 15:01:37.235445 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.235346 2561 manager.go:216] Machine: {Timestamp:2026-06-10 15:01:37.233990265 +0000 UTC m=+0.366283050 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100192 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22d6451ffda783ecc51161a369ee53 SystemUUID:ec22d645-1ffd-a783-ecc5-1161a369ee53 BootID:7d264db2-f14d-4330-b842-38bba09af3a9 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:76:48:19:46:8d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:76:48:19:46:8d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:fb:8f:d2:68:e5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 10 15:01:37.235445 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.235439 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 10 15:01:37.235550 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.235498 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 10 15:01:37.238123 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.238098 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 10 15:01:37.238249 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.238125 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-85.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 10 15:01:37.239150 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.239140 2561 topology_manager.go:138] "Creating topology manager with none policy" Jun 10 15:01:37.239184 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.239153 2561 container_manager_linux.go:309] "Creating device plugin manager" Jun 10 15:01:37.239184 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.239161 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 15:01:37.239762 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.239748 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 15:01:37.240163 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.240152 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 15:01:37.241056 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.241047 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 10 15:01:37.245167 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.245155 2561 state_mem.go:36] "Initialized new in-memory state store" Jun 10 15:01:37.245263 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.245255 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 10 15:01:37.245294 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.245281 2561 kubelet.go:486] "Attempting to sync node with API server" Jun 10 15:01:37.245294 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.245290 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 10 15:01:37.245347 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.245302 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 10 15:01:37.245347 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.245310 2561 kubelet.go:398] "Adding apiserver pod source" Jun 10 15:01:37.245347 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.245324 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 10 15:01:37.245604 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.245586 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 15:01:37.245676 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.245611 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 15:01:37.247918 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.247897 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 10 15:01:37.249661 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.249649 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 10 15:01:37.249704 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.249678 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 10 15:01:37.251009 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.250995 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 10 15:01:37.251044 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251015 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 10 15:01:37.251044 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251021 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 10 15:01:37.251044 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251026 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 10 15:01:37.251044 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251035 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 10 15:01:37.251044 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251041 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 10 15:01:37.251188 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251050 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 10 15:01:37.251188 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251059 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 10 15:01:37.251188 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251068 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 10 15:01:37.251188 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251073 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 10 15:01:37.251188 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251081 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 10 15:01:37.251188 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.251086 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 10 15:01:37.252456 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.252445 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 10 15:01:37.252456 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.252456 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 10 15:01:37.255581 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.255452 2561 server.go:1268] "Started kubelet" Jun 10 15:01:37.255581 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.255540 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 10 15:01:37.255739 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.255520 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 10 15:01:37.255739 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.255614 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 10 15:01:37.256324 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.256287 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 10 15:01:37.256441 ip-10-0-129-85 systemd[1]: Started Kubernetes Kubelet. Jun 10 15:01:37.256970 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.256553 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 10 15:01:37.256970 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.256587 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 10 15:01:37.260111 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.259906 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 10 15:01:37.262458 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.262436 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 10 15:01:37.262547 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.262466 2561 server.go:310] "Adding debug handlers to kubelet server" Jun 10 15:01:37.264997 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.264108 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-85.ec2.internal.18b7c0603f0b6739 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-85.ec2.internal,UID:ip-10-0-129-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-85.ec2.internal,},FirstTimestamp:2026-06-10 15:01:37.255425849 +0000 UTC m=+0.387718635,LastTimestamp:2026-06-10 15:01:37.255425849 +0000 UTC m=+0.387718635,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-85.ec2.internal,}" Jun 10 15:01:37.265737 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.265721 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 10 15:01:37.265737 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.265733 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 10 15:01:37.265858 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.265791 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:37.266424 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266406 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 10 15:01:37.266424 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266406 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 10 15:01:37.266557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266434 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 10 15:01:37.266557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266528 2561 factory.go:55] Registering systemd factory Jun 10 15:01:37.266557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266557 2561 factory.go:223] Registration of the systemd container factory successfully Jun 10 15:01:37.266736 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266556 2561 reconstruct.go:97] "Volume reconstruction finished" Jun 10 15:01:37.266736 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266599 2561 reconciler.go:29] "Reconciler: start to sync state" Jun 10 15:01:37.266835 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266766 2561 factory.go:153] Registering CRI-O factory Jun 10 15:01:37.266835 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266781 2561 factory.go:223] Registration of the crio container factory successfully Jun 10 15:01:37.266835 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266823 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 10 15:01:37.266993 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266848 2561 factory.go:103] Registering Raw factory Jun 10 15:01:37.266993 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.266865 2561 manager.go:1196] Started watching for new ooms in manager Jun 10 15:01:37.267268 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.267250 2561 manager.go:349] Starting recovery of all containers Jun 10 15:01:37.276247 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.276224 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 10 15:01:37.276538 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.276510 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 10 15:01:37.278411 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.278395 2561 manager.go:354] Recovery completed Jun 10 15:01:37.282388 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.282376 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 15:01:37.284257 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.284195 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientMemory" Jun 10 15:01:37.284257 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.284225 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 15:01:37.284257 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.284241 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientPID" Jun 10 15:01:37.284804 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.284790 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 10 15:01:37.284804 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.284801 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 10 15:01:37.284804 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.284792 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 15:01:37.284986 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.284815 2561 state_mem.go:36] "Initialized new in-memory state store" Jun 10 15:01:37.285449 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.285435 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientMemory" Jun 10 15:01:37.285500 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.285460 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 15:01:37.285500 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.285470 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientPID" Jun 10 15:01:37.286640 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.286625 2561 policy_none.go:49] "None policy: Start" Jun 10 15:01:37.286713 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.286648 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 10 15:01:37.286713 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.286671 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 10 15:01:37.288401 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.288389 2561 policy_none.go:47] "Start" Jun 10 15:01:37.295837 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.295779 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-85.ec2.internal.18b7c06040c29350 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-85.ec2.internal,UID:ip-10-0-129-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-85.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-85.ec2.internal,},FirstTimestamp:2026-06-10 15:01:37.28420744 +0000 UTC m=+0.416500227,LastTimestamp:2026-06-10 15:01:37.28420744 +0000 UTC m=+0.416500227,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-85.ec2.internal,}" Jun 10 15:01:37.304467 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.304452 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cj72p" Jun 10 15:01:37.308275 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.308216 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-85.ec2.internal.18b7c06040c2f1a8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-85.ec2.internal,UID:ip-10-0-129-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-85.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-85.ec2.internal,},FirstTimestamp:2026-06-10 15:01:37.284231592 +0000 UTC m=+0.416524380,LastTimestamp:2026-06-10 15:01:37.284231592 +0000 UTC m=+0.416524380,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-85.ec2.internal,}" Jun 10 15:01:37.316655 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.316636 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cj72p" Jun 10 15:01:37.316744 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.316727 2561 manager.go:337] "Starting Device Plugin manager" Jun 10 15:01:37.316791 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.316771 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 10 15:01:37.316791 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.316784 2561 server.go:87] "Starting device plugin registration server" Jun 10 15:01:37.317033 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.317022 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 10 15:01:37.317084 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.317034 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 10 15:01:37.317282 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.317141 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 10 15:01:37.317282 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.317232 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 10 15:01:37.317282 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.317242 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 10 15:01:37.317627 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.317610 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 10 15:01:37.317697 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.317653 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:37.319753 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.319694 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-85.ec2.internal.18b7c06040c32ec4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-85.ec2.internal,UID:ip-10-0-129-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-85.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-85.ec2.internal,},FirstTimestamp:2026-06-10 15:01:37.284247236 +0000 UTC m=+0.416540025,LastTimestamp:2026-06-10 15:01:37.284247236 +0000 UTC m=+0.416540025,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-85.ec2.internal,}" Jun 10 15:01:37.335863 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.335842 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 10 15:01:37.352435 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.336911 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 10 15:01:37.352435 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.336954 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 10 15:01:37.352435 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.336973 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jun 10 15:01:37.352435 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.336999 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 10 15:01:37.352435 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.339184 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 15:01:37.418008 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.417980 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 15:01:37.418804 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.418790 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientMemory" Jun 10 15:01:37.418898 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.418822 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 15:01:37.418898 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.418836 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientPID" Jun 10 15:01:37.418898 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.418861 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.426358 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.426344 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.426420 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.426363 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-85.ec2.internal\": node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:37.437869 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.437822 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal"] Jun 10 15:01:37.437960 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.437873 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 15:01:37.438630 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.438611 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientMemory" Jun 10 15:01:37.438735 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.438643 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 15:01:37.438735 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.438654 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientPID" Jun 10 15:01:37.440446 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.440414 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 15:01:37.440579 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.440565 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.440636 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.440599 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 15:01:37.441135 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.441119 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientMemory" Jun 10 15:01:37.441135 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.441129 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientMemory" Jun 10 15:01:37.441280 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.441147 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 15:01:37.441280 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.441160 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientPID" Jun 10 15:01:37.441280 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.441150 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 15:01:37.441393 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.441293 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientPID" Jun 10 15:01:37.443163 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.443150 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.443211 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.443173 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 15:01:37.443844 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.443828 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientMemory" Jun 10 15:01:37.443955 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.443854 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 15:01:37.443955 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.443863 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeHasSufficientPID" Jun 10 15:01:37.457909 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.457886 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:37.468577 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.468554 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-85.ec2.internal\" not found" node="ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.473185 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.473170 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-85.ec2.internal\" not found" node="ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.558622 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.558602 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:37.567688 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.567666 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c9a9f2b5539e7a75832f63deb43d54cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal\" (UID: \"c9a9f2b5539e7a75832f63deb43d54cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.567757 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.567701 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9a9f2b5539e7a75832f63deb43d54cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal\" (UID: \"c9a9f2b5539e7a75832f63deb43d54cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.567757 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.567735 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbfeb1994c2bac4a8dfa92f1e1823c15-config\") pod \"kube-apiserver-proxy-ip-10-0-129-85.ec2.internal\" (UID: \"bbfeb1994c2bac4a8dfa92f1e1823c15\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.658883 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.658852 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:37.668180 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.668161 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c9a9f2b5539e7a75832f63deb43d54cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal\" (UID: \"c9a9f2b5539e7a75832f63deb43d54cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.668225 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.668184 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9a9f2b5539e7a75832f63deb43d54cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal\" (UID: \"c9a9f2b5539e7a75832f63deb43d54cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.668225 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.668203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbfeb1994c2bac4a8dfa92f1e1823c15-config\") pod \"kube-apiserver-proxy-ip-10-0-129-85.ec2.internal\" (UID: \"bbfeb1994c2bac4a8dfa92f1e1823c15\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.668325 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.668243 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbfeb1994c2bac4a8dfa92f1e1823c15-config\") pod \"kube-apiserver-proxy-ip-10-0-129-85.ec2.internal\" (UID: \"bbfeb1994c2bac4a8dfa92f1e1823c15\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.668325 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.668265 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c9a9f2b5539e7a75832f63deb43d54cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal\" (UID: \"c9a9f2b5539e7a75832f63deb43d54cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.668325 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.668290 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9a9f2b5539e7a75832f63deb43d54cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal\" (UID: \"c9a9f2b5539e7a75832f63deb43d54cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.759601 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.759546 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:37.773003 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.772981 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.777992 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:37.777975 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal" Jun 10 15:01:37.860658 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.860631 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:37.961088 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:37.961063 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:38.061644 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:38.061585 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:38.162077 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:38.162056 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:38.204615 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.204590 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 10 15:01:38.205182 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.204760 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 10 15:01:38.262118 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:38.262093 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:38.320952 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.320873 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-09 14:56:37 +0000 UTC" deadline="2028-02-21 05:21:19.657147964 +0000 UTC" Jun 10 15:01:38.320952 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.320907 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14894h19m41.336245427s" Jun 10 15:01:38.363122 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:38.363100 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-85.ec2.internal\" not found" Jun 10 15:01:38.441323 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:38.441293 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a9f2b5539e7a75832f63deb43d54cb.slice/crio-bf0bf08913733b024463673894548ea73782c43f7b26504a2c733306f234a1e6 WatchSource:0}: Error finding container bf0bf08913733b024463673894548ea73782c43f7b26504a2c733306f234a1e6: Status 404 returned error can't find the container with id bf0bf08913733b024463673894548ea73782c43f7b26504a2c733306f234a1e6 Jun 10 15:01:38.441716 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:38.441698 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbfeb1994c2bac4a8dfa92f1e1823c15.slice/crio-a295b2ae0c80ec4c66e65b0a510c56dc69ecac1a1676235c75c80c52240e7dd4 WatchSource:0}: Error finding container a295b2ae0c80ec4c66e65b0a510c56dc69ecac1a1676235c75c80c52240e7dd4: Status 404 returned error can't find the container with id a295b2ae0c80ec4c66e65b0a510c56dc69ecac1a1676235c75c80c52240e7dd4 Jun 10 15:01:38.448521 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.448507 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 15:01:38.449228 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.449215 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 15:01:38.466293 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.466277 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal" Jun 10 15:01:38.482497 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.482482 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 15:01:38.485083 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.485072 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" Jun 10 15:01:38.499794 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.499779 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 15:01:38.556802 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.556786 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 15:01:38.619633 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:38.619586 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 15:01:39.246753 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.246727 2561 apiserver.go:52] "Watching apiserver" Jun 10 15:01:39.259430 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.259410 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 10 15:01:39.259752 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.259731 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal","openshift-multus/multus-additional-cni-plugins-v8gqd","openshift-multus/multus-wqtc6","openshift-multus/network-metrics-daemon-msxf9","openshift-network-diagnostics/network-check-target-2mbrv","openshift-network-operator/iptables-alerter-nrp6w","openshift-ovn-kubernetes/ovnkube-node-8pkch","kube-system/global-pull-secret-syncer-fxmn5","kube-system/konnectivity-agent-c89rn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9","openshift-cluster-node-tuning-operator/tuned-l575g","openshift-dns/node-resolver-djm7n","openshift-image-registry/node-ca-kvf47"] Jun 10 15:01:39.262139 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.262120 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.263840 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.263821 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.265688 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.265665 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.266036 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.266015 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 10 15:01:39.267222 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.267189 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.267342 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.267261 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 10 15:01:39.267424 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.267398 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 10 15:01:39.267743 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.267677 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 10 15:01:39.267861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.267789 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 10 15:01:39.268239 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.268224 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7j6mq\"" Jun 10 15:01:39.268351 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.268277 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 10 15:01:39.268631 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.268612 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nqcwb\"" Jun 10 15:01:39.269144 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.268915 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 10 15:01:39.269232 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.269190 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:39.269299 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.269254 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:39.271187 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.271130 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 10 15:01:39.271313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.271145 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 10 15:01:39.272086 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.272066 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:39.272202 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.272125 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:39.272386 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.272369 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 10 15:01:39.272470 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.272451 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5g74s\"" Jun 10 15:01:39.272648 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.272532 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-csb6l\"" Jun 10 15:01:39.272648 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.272378 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 10 15:01:39.272648 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.272421 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 10 15:01:39.275453 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275273 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.275453 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.275331 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:39.275593 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275483 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-os-release\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.275593 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-socket-dir-parent\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.275593 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275540 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.275593 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275566 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-run-k8s-cni-cncf-io\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.275593 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275591 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-run-netns\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.275832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275617 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-conf-dir\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.275832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275642 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e512dd9b-89b7-4bb0-abef-1d717f999ddc-hosts-file\") pod \"node-resolver-djm7n\" (UID: \"e512dd9b-89b7-4bb0-abef-1d717f999ddc\") " pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.275832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275667 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-system-cni-dir\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.275832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275693 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-cnibin\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.275832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275719 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.275832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275747 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-cni-dir\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.275832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275771 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-var-lib-cni-bin\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.275832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275795 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-var-lib-kubelet\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.275832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275820 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-hostroot\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74e4cc5f-e66f-4eb2-84af-b6ce55aa4824-host-slash\") pod \"iptables-alerter-nrp6w\" (UID: \"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824\") " pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275867 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgd7t\" (UniqueName: \"kubernetes.io/projected/74e4cc5f-e66f-4eb2-84af-b6ce55aa4824-kube-api-access-jgd7t\") pod \"iptables-alerter-nrp6w\" (UID: \"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824\") " pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-daemon-config\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.275922 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/74e4cc5f-e66f-4eb2-84af-b6ce55aa4824-iptables-alerter-script\") pod \"iptables-alerter-nrp6w\" (UID: \"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824\") " pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276002 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e512dd9b-89b7-4bb0-abef-1d717f999ddc-tmp-dir\") pod \"node-resolver-djm7n\" (UID: \"e512dd9b-89b7-4bb0-abef-1d717f999ddc\") " pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276044 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prt7x\" (UniqueName: \"kubernetes.io/projected/e512dd9b-89b7-4bb0-abef-1d717f999ddc-kube-api-access-prt7x\") pod \"node-resolver-djm7n\" (UID: \"e512dd9b-89b7-4bb0-abef-1d717f999ddc\") " pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276086 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276146 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-cnibin\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276196 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-cni-binary-copy\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276232 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-var-lib-cni-multus\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276261 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.276868 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-os-release\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276868 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276308 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-run-multus-certs\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276868 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276346 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-etc-kubernetes\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276868 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276399 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwqmf\" (UniqueName: \"kubernetes.io/projected/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-kube-api-access-zwqmf\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.276868 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276423 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-system-cni-dir\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276868 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276448 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww7pf\" (UniqueName: \"kubernetes.io/projected/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-kube-api-access-ww7pf\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.276868 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.276858 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.278588 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.278511 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.280331 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.280309 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:01:39.281981 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.281962 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.283286 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.283265 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hgzgs\"" Jun 10 15:01:39.283553 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.283458 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 10 15:01:39.283753 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.283655 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 10 15:01:39.283753 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.283674 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 10 15:01:39.283753 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.283726 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fthqm\"" Jun 10 15:01:39.283913 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.283772 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.283913 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.283881 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 10 15:01:39.284396 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.284356 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 10 15:01:39.284908 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.284809 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 10 15:01:39.285192 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285144 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 10 15:01:39.285192 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285185 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 10 15:01:39.285447 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285284 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 10 15:01:39.285447 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285398 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 10 15:01:39.285447 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285414 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 10 15:01:39.285676 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285525 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cwnm5\"" Jun 10 15:01:39.285676 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285580 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 10 15:01:39.285676 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285660 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7lxdf\"" Jun 10 15:01:39.285866 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285778 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 10 15:01:39.286149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.285969 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 10 15:01:39.289216 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.289180 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ldp6d\"" Jun 10 15:01:39.289304 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.289278 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 10 15:01:39.290198 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.290132 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 10 15:01:39.291442 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.291268 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 15:01:39.305504 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.305479 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v6t42" Jun 10 15:01:39.317545 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.317525 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v6t42" Jun 10 15:01:39.341651 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.341584 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" event={"ID":"c9a9f2b5539e7a75832f63deb43d54cb","Type":"ContainerStarted","Data":"bf0bf08913733b024463673894548ea73782c43f7b26504a2c733306f234a1e6"} Jun 10 15:01:39.343055 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.343029 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal" event={"ID":"bbfeb1994c2bac4a8dfa92f1e1823c15","Type":"ContainerStarted","Data":"a295b2ae0c80ec4c66e65b0a510c56dc69ecac1a1676235c75c80c52240e7dd4"} Jun 10 15:01:39.368520 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.368501 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 10 15:01:39.376663 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376641 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-system-cni-dir\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.376764 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376672 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.376764 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376698 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-cni-dir\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.376764 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376723 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-var-lib-kubelet\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.376764 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-hostroot\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.376764 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376756 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-system-cni-dir\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376770 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74e4cc5f-e66f-4eb2-84af-b6ce55aa4824-host-slash\") pod \"iptables-alerter-nrp6w\" (UID: \"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824\") " pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376802 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-run-openvswitch\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376827 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376839 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-cni-dir\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376844 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-kubernetes\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376870 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prt7x\" (UniqueName: \"kubernetes.io/projected/e512dd9b-89b7-4bb0-abef-1d717f999ddc-kube-api-access-prt7x\") pod \"node-resolver-djm7n\" (UID: \"e512dd9b-89b7-4bb0-abef-1d717f999ddc\") " pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376881 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-var-lib-kubelet\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376886 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376911 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-cni-binary-copy\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.376955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-var-lib-cni-multus\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377010 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-kubelet\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377020 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-hostroot\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377033 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-var-lib-cni-multus\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377034 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e836d9ae-14e4-4798-9250-669c90f1de68-ovn-node-metrics-cert\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.377067 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377071 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74e4cc5f-e66f-4eb2-84af-b6ce55aa4824-host-slash\") pod \"iptables-alerter-nrp6w\" (UID: \"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824\") " pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377097 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c4b47316-b098-43dc-aa43-973bca48c3c2-dbus\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377131 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-host\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377161 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377186 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-os-release\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e836d9ae-14e4-4798-9250-669c90f1de68-env-overrides\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-system-cni-dir\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377293 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-system-cni-dir\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377300 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377330 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377305 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-os-release\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377381 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7qfb\" (UniqueName: \"kubernetes.io/projected/0d107589-1c68-4bf6-a05a-31045a73f204-kube-api-access-x7qfb\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377432 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-run\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377460 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53f26c7f-fc18-4658-b7a7-a29a2d392b25-tmp\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-cni-bin\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377508 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-cni-binary-copy\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377511 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91cc19fa-ee08-4fb7-bd80-84897bff8842-agent-certs\") pod \"konnectivity-agent-c89rn\" (UID: \"91cc19fa-ee08-4fb7-bd80-84897bff8842\") " pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:01:39.377763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377554 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-sys-fs\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377608 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-run-netns\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377634 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-sysconfig\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377678 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-tuned\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377681 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-run-netns\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377683 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377702 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-conf-dir\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377738 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-etc-openvswitch\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377746 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377752 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91cc19fa-ee08-4fb7-bd80-84897bff8842-konnectivity-ca\") pod \"konnectivity-agent-c89rn\" (UID: \"91cc19fa-ee08-4fb7-bd80-84897bff8842\") " pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377759 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-conf-dir\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377784 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-var-lib-kubelet\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377790 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377810 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-cnibin\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377834 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-cnibin\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377844 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-var-lib-cni-bin\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.378618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377873 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-daemon-config\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377900 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-slash\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377956 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-var-lib-cni-bin\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.377985 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6kc9\" (UniqueName: \"kubernetes.io/projected/e836d9ae-14e4-4798-9250-669c90f1de68-kube-api-access-d6kc9\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378011 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-device-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378033 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c4b47316-b098-43dc-aa43-973bca48c3c2-kubelet-config\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378054 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-modprobe-d\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378092 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgd7t\" (UniqueName: \"kubernetes.io/projected/74e4cc5f-e66f-4eb2-84af-b6ce55aa4824-kube-api-access-jgd7t\") pod \"iptables-alerter-nrp6w\" (UID: \"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824\") " pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378116 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378151 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-systemd\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378178 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/74e4cc5f-e66f-4eb2-84af-b6ce55aa4824-iptables-alerter-script\") pod \"iptables-alerter-nrp6w\" (UID: \"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824\") " pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e512dd9b-89b7-4bb0-abef-1d717f999ddc-tmp-dir\") pod \"node-resolver-djm7n\" (UID: \"e512dd9b-89b7-4bb0-abef-1d717f999ddc\") " pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378226 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-cnibin\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378249 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-run-netns\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378273 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-var-lib-openvswitch\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378298 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-node-log\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378321 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbf6x\" (UniqueName: \"kubernetes.io/projected/49563c45-0b4c-404d-b5fd-46b64abfa6a8-kube-api-access-wbf6x\") pod \"node-ca-kvf47\" (UID: \"49563c45-0b4c-404d-b5fd-46b64abfa6a8\") " pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.379433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378348 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-run-multus-certs\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378372 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-etc-kubernetes\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378383 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-daemon-config\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378396 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-etc-selinux\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378423 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378447 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-sysctl-d\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378471 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-lib-modules\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwqmf\" (UniqueName: \"kubernetes.io/projected/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-kube-api-access-zwqmf\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378521 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww7pf\" (UniqueName: \"kubernetes.io/projected/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-kube-api-access-ww7pf\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378560 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-run-ovn-kubernetes\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378586 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e836d9ae-14e4-4798-9250-669c90f1de68-ovnkube-script-lib\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378611 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-os-release\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378636 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-socket-dir-parent\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378659 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-run-systemd\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378664 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e512dd9b-89b7-4bb0-abef-1d717f999ddc-tmp-dir\") pod \"node-resolver-djm7n\" (UID: \"e512dd9b-89b7-4bb0-abef-1d717f999ddc\") " pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378690 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-run-ovn\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378716 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/74e4cc5f-e66f-4eb2-84af-b6ce55aa4824-iptables-alerter-script\") pod \"iptables-alerter-nrp6w\" (UID: \"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824\") " pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.380201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378741 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-log-socket\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378753 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-cnibin\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378773 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-cni-netd\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378798 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-run-multus-certs\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378838 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e836d9ae-14e4-4798-9250-669c90f1de68-ovnkube-config\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378844 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-etc-kubernetes\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378863 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-socket-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378888 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-run-k8s-cni-cncf-io\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378911 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-registration-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378954 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb8mc\" (UniqueName: \"kubernetes.io/projected/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-kube-api-access-fb8mc\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.378980 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379017 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-os-release\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379047 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-sys\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379062 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-multus-socket-dir-parent\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379075 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwtsv\" (UniqueName: \"kubernetes.io/projected/53f26c7f-fc18-4658-b7a7-a29a2d392b25-kube-api-access-cwtsv\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379099 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-host-run-k8s-cni-cncf-io\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379101 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49563c45-0b4c-404d-b5fd-46b64abfa6a8-host\") pod \"node-ca-kvf47\" (UID: \"49563c45-0b4c-404d-b5fd-46b64abfa6a8\") " pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.380922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379126 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49563c45-0b4c-404d-b5fd-46b64abfa6a8-serviceca\") pod \"node-ca-kvf47\" (UID: \"49563c45-0b4c-404d-b5fd-46b64abfa6a8\") " pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.381584 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379164 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-systemd-units\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.381584 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379196 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-sysctl-conf\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.381584 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e512dd9b-89b7-4bb0-abef-1d717f999ddc-hosts-file\") pod \"node-resolver-djm7n\" (UID: \"e512dd9b-89b7-4bb0-abef-1d717f999ddc\") " pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.381584 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.379336 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e512dd9b-89b7-4bb0-abef-1d717f999ddc-hosts-file\") pod \"node-resolver-djm7n\" (UID: \"e512dd9b-89b7-4bb0-abef-1d717f999ddc\") " pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.390238 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.390212 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 10 15:01:39.393070 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.393048 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prt7x\" (UniqueName: \"kubernetes.io/projected/e512dd9b-89b7-4bb0-abef-1d717f999ddc-kube-api-access-prt7x\") pod \"node-resolver-djm7n\" (UID: \"e512dd9b-89b7-4bb0-abef-1d717f999ddc\") " pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.415171 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.415149 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwqmf\" (UniqueName: \"kubernetes.io/projected/b73ecdfe-8356-4a0a-872b-1116ee1a95b9-kube-api-access-zwqmf\") pod \"multus-additional-cni-plugins-v8gqd\" (UID: \"b73ecdfe-8356-4a0a-872b-1116ee1a95b9\") " pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.428428 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.428404 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww7pf\" (UniqueName: \"kubernetes.io/projected/72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947-kube-api-access-ww7pf\") pod \"multus-wqtc6\" (UID: \"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947\") " pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.432516 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.432494 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgd7t\" (UniqueName: \"kubernetes.io/projected/74e4cc5f-e66f-4eb2-84af-b6ce55aa4824-kube-api-access-jgd7t\") pod \"iptables-alerter-nrp6w\" (UID: \"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824\") " pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.479828 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.479798 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-kubelet\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.479946 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.479840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e836d9ae-14e4-4798-9250-669c90f1de68-ovn-node-metrics-cert\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.479946 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.479867 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c4b47316-b098-43dc-aa43-973bca48c3c2-dbus\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.479946 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.479893 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-host\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.479946 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.479901 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-kubelet\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.479954 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e836d9ae-14e4-4798-9250-669c90f1de68-env-overrides\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.479984 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480012 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7qfb\" (UniqueName: \"kubernetes.io/projected/0d107589-1c68-4bf6-a05a-31045a73f204-kube-api-access-x7qfb\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480036 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-run\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480044 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c4b47316-b098-43dc-aa43-973bca48c3c2-dbus\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53f26c7f-fc18-4658-b7a7-a29a2d392b25-tmp\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480086 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-cni-bin\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480096 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91cc19fa-ee08-4fb7-bd80-84897bff8842-agent-certs\") pod \"konnectivity-agent-c89rn\" (UID: \"91cc19fa-ee08-4fb7-bd80-84897bff8842\") " pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480138 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-sys-fs\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.480149 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480148 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-host\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480166 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-sysconfig\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480191 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-tuned\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480219 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-etc-openvswitch\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480245 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-cni-bin\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480249 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91cc19fa-ee08-4fb7-bd80-84897bff8842-konnectivity-ca\") pod \"konnectivity-agent-c89rn\" (UID: \"91cc19fa-ee08-4fb7-bd80-84897bff8842\") " pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480292 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-var-lib-kubelet\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480323 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-slash\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6kc9\" (UniqueName: \"kubernetes.io/projected/e836d9ae-14e4-4798-9250-669c90f1de68-kube-api-access-d6kc9\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480371 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-device-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480395 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c4b47316-b098-43dc-aa43-973bca48c3c2-kubelet-config\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480419 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-modprobe-d\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480444 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480466 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-systemd\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480493 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-run-netns\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480516 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-var-lib-openvswitch\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480541 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-node-log\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.480667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480551 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e836d9ae-14e4-4798-9250-669c90f1de68-env-overrides\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480564 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbf6x\" (UniqueName: \"kubernetes.io/projected/49563c45-0b4c-404d-b5fd-46b64abfa6a8-kube-api-access-wbf6x\") pod \"node-ca-kvf47\" (UID: \"49563c45-0b4c-404d-b5fd-46b64abfa6a8\") " pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480592 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-etc-selinux\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480615 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480606 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-device-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480677 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c4b47316-b098-43dc-aa43-973bca48c3c2-kubelet-config\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480775 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-modprobe-d\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481001 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-systemd\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480841 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-run\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480864 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-sys-fs\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480944 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-sysconfig\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481038 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-node-log\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481045 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-var-lib-openvswitch\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.480787 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91cc19fa-ee08-4fb7-bd80-84897bff8842-konnectivity-ca\") pod \"konnectivity-agent-c89rn\" (UID: \"91cc19fa-ee08-4fb7-bd80-84897bff8842\") " pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481090 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-etc-openvswitch\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481131 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-slash\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481168 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-sysctl-d\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.481495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481184 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-etc-selinux\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481212 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-lib-modules\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481229 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-var-lib-kubelet\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481241 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-run-ovn-kubernetes\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481264 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-run-netns\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481266 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e836d9ae-14e4-4798-9250-669c90f1de68-ovnkube-script-lib\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481303 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-run-systemd\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481327 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-run-ovn\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481349 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-log-socket\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481368 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-cni-netd\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481390 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e836d9ae-14e4-4798-9250-669c90f1de68-ovnkube-config\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481403 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-lib-modules\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481414 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-socket-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481439 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-registration-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481447 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-run-systemd\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481463 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb8mc\" (UniqueName: \"kubernetes.io/projected/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-kube-api-access-fb8mc\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481488 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.482313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481492 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-sysctl-d\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481512 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-sys\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481533 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-run-ovn-kubernetes\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481536 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwtsv\" (UniqueName: \"kubernetes.io/projected/53f26c7f-fc18-4658-b7a7-a29a2d392b25-kube-api-access-cwtsv\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481568 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49563c45-0b4c-404d-b5fd-46b64abfa6a8-host\") pod \"node-ca-kvf47\" (UID: \"49563c45-0b4c-404d-b5fd-46b64abfa6a8\") " pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481591 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49563c45-0b4c-404d-b5fd-46b64abfa6a8-serviceca\") pod \"node-ca-kvf47\" (UID: \"49563c45-0b4c-404d-b5fd-46b64abfa6a8\") " pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-systemd-units\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-sysctl-conf\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481683 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-run-openvswitch\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481709 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481732 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-kubernetes\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481734 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e836d9ae-14e4-4798-9250-669c90f1de68-ovnkube-script-lib\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481800 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-run-openvswitch\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481818 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-socket-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481413 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-run-ovn\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481875 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-registration-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481887 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e836d9ae-14e4-4798-9250-669c90f1de68-ovnkube-config\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481898 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-sysctl-conf\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.482906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481970 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-sys\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.481976 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-log-socket\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.482032 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-kubernetes\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.482175 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49563c45-0b4c-404d-b5fd-46b64abfa6a8-serviceca\") pod \"node-ca-kvf47\" (UID: \"49563c45-0b4c-404d-b5fd-46b64abfa6a8\") " pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.482219 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-systemd-units\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.482254 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49563c45-0b4c-404d-b5fd-46b64abfa6a8-host\") pod \"node-ca-kvf47\" (UID: \"49563c45-0b4c-404d-b5fd-46b64abfa6a8\") " pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.482378 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0d107589-1c68-4bf6-a05a-31045a73f204-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.482536 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.482613 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:01:39.982591687 +0000 UTC m=+3.114884481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.483050 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.483100 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret podName:c4b47316-b098-43dc-aa43-973bca48c3c2 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:39.983085776 +0000 UTC m=+3.115378555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret") pod "global-pull-secret-syncer-fxmn5" (UID: "c4b47316-b098-43dc-aa43-973bca48c3c2") : object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:39.483535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.483282 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/53f26c7f-fc18-4658-b7a7-a29a2d392b25-etc-tuned\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.484115 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.483979 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e836d9ae-14e4-4798-9250-669c90f1de68-host-cni-netd\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.484610 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.484554 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e836d9ae-14e4-4798-9250-669c90f1de68-ovn-node-metrics-cert\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.484610 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.484575 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53f26c7f-fc18-4658-b7a7-a29a2d392b25-tmp\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.485624 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.485549 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91cc19fa-ee08-4fb7-bd80-84897bff8842-agent-certs\") pod \"konnectivity-agent-c89rn\" (UID: \"91cc19fa-ee08-4fb7-bd80-84897bff8842\") " pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:01:39.494335 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.494293 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbf6x\" (UniqueName: \"kubernetes.io/projected/49563c45-0b4c-404d-b5fd-46b64abfa6a8-kube-api-access-wbf6x\") pod \"node-ca-kvf47\" (UID: \"49563c45-0b4c-404d-b5fd-46b64abfa6a8\") " pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.495635 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.495617 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 15:01:39.495706 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.495644 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 15:01:39.495706 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.495658 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kzhdj for pod openshift-network-diagnostics/network-check-target-2mbrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:39.495816 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.495711 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj podName:07d53553-c4d6-4df1-a2f1-8d6a74ac5b72 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:39.99569446 +0000 UTC m=+3.127987249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kzhdj" (UniqueName: "kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj") pod "network-check-target-2mbrv" (UID: "07d53553-c4d6-4df1-a2f1-8d6a74ac5b72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:39.496270 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.496252 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwtsv\" (UniqueName: \"kubernetes.io/projected/53f26c7f-fc18-4658-b7a7-a29a2d392b25-kube-api-access-cwtsv\") pod \"tuned-l575g\" (UID: \"53f26c7f-fc18-4658-b7a7-a29a2d392b25\") " pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.496717 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.496659 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7qfb\" (UniqueName: \"kubernetes.io/projected/0d107589-1c68-4bf6-a05a-31045a73f204-kube-api-access-x7qfb\") pod \"aws-ebs-csi-driver-node-rnrg9\" (UID: \"0d107589-1c68-4bf6-a05a-31045a73f204\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.498260 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.498190 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb8mc\" (UniqueName: \"kubernetes.io/projected/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-kube-api-access-fb8mc\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:39.504179 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.504159 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6kc9\" (UniqueName: \"kubernetes.io/projected/e836d9ae-14e4-4798-9250-669c90f1de68-kube-api-access-d6kc9\") pod \"ovnkube-node-8pkch\" (UID: \"e836d9ae-14e4-4798-9250-669c90f1de68\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.574939 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.574908 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nrp6w" Jun 10 15:01:39.582094 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.582074 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-djm7n" Jun 10 15:01:39.592864 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.592848 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" Jun 10 15:01:39.603808 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.603790 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wqtc6" Jun 10 15:01:39.610629 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.610608 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kvf47" Jun 10 15:01:39.617434 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.617404 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:01:39.624080 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.624063 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:01:39.630828 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.630808 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" Jun 10 15:01:39.635841 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.635819 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l575g" Jun 10 15:01:39.692353 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.692329 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 15:01:39.986061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.986003 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:39.986061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:39.986046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:39.986216 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.986131 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:39.986216 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.986176 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret podName:c4b47316-b098-43dc-aa43-973bca48c3c2 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:40.986163298 +0000 UTC m=+4.118456074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret") pod "global-pull-secret-syncer-fxmn5" (UID: "c4b47316-b098-43dc-aa43-973bca48c3c2") : object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:39.986216 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.986132 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:39.986311 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:39.986245 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:01:40.986230301 +0000 UTC m=+4.118523073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:40.087036 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:40.087009 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:40.087216 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:40.087197 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 15:01:40.087270 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:40.087226 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 15:01:40.087270 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:40.087243 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kzhdj for pod openshift-network-diagnostics/network-check-target-2mbrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:40.087361 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:40.087314 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj podName:07d53553-c4d6-4df1-a2f1-8d6a74ac5b72 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:41.087294033 +0000 UTC m=+4.219586824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzhdj" (UniqueName: "kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj") pod "network-check-target-2mbrv" (UID: "07d53553-c4d6-4df1-a2f1-8d6a74ac5b72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:40.319718 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:40.319685 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 14:56:39 +0000 UTC" deadline="2028-02-19 00:12:14.433852794 +0000 UTC" Jun 10 15:01:40.319718 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:40.319713 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14841h10m34.114143153s" Jun 10 15:01:40.328266 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:40.328240 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e4cc5f_e66f_4eb2_84af_b6ce55aa4824.slice/crio-81adfb67e66a1fc0e74375aede1fafdf1694b0ea4bf9bc95ea818313aef79db7 WatchSource:0}: Error finding container 81adfb67e66a1fc0e74375aede1fafdf1694b0ea4bf9bc95ea818313aef79db7: Status 404 returned error can't find the container with id 81adfb67e66a1fc0e74375aede1fafdf1694b0ea4bf9bc95ea818313aef79db7 Jun 10 15:01:40.328878 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:40.328858 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f26c7f_fc18_4658_b7a7_a29a2d392b25.slice/crio-2ddb61d3f03b317a8996b02d8fef18973459d77cc23ea39ca5c6055f70d59600 WatchSource:0}: Error finding container 2ddb61d3f03b317a8996b02d8fef18973459d77cc23ea39ca5c6055f70d59600: Status 404 returned error can't find the container with id 2ddb61d3f03b317a8996b02d8fef18973459d77cc23ea39ca5c6055f70d59600 Jun 10 15:01:40.344829 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:40.344806 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l575g" event={"ID":"53f26c7f-fc18-4658-b7a7-a29a2d392b25","Type":"ContainerStarted","Data":"2ddb61d3f03b317a8996b02d8fef18973459d77cc23ea39ca5c6055f70d59600"} Jun 10 15:01:40.346396 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:40.346370 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nrp6w" event={"ID":"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824","Type":"ContainerStarted","Data":"81adfb67e66a1fc0e74375aede1fafdf1694b0ea4bf9bc95ea818313aef79db7"} Jun 10 15:01:40.346627 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:40.346594 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73ecdfe_8356_4a0a_872b_1116ee1a95b9.slice/crio-68fb275715da64db310152d3edfdcfc4d526836886349853b7f4d01f8abdc844 WatchSource:0}: Error finding container 68fb275715da64db310152d3edfdcfc4d526836886349853b7f4d01f8abdc844: Status 404 returned error can't find the container with id 68fb275715da64db310152d3edfdcfc4d526836886349853b7f4d01f8abdc844 Jun 10 15:01:40.350199 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:40.350180 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f1e4b5_60f0_41a4_b9dd_2a83c4dfc947.slice/crio-dc6d70bc7e9e0cd2ebc6b0062bf9a7e812bb2a51c45fb298a5a3d2d0196de06e WatchSource:0}: Error finding container dc6d70bc7e9e0cd2ebc6b0062bf9a7e812bb2a51c45fb298a5a3d2d0196de06e: Status 404 returned error can't find the container with id dc6d70bc7e9e0cd2ebc6b0062bf9a7e812bb2a51c45fb298a5a3d2d0196de06e Jun 10 15:01:40.353685 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:40.353661 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode512dd9b_89b7_4bb0_abef_1d717f999ddc.slice/crio-f68744e9b958e5abe39b5abc5b45ea0dcf05bb0861da9ede0081d48727a5c57b WatchSource:0}: Error finding container f68744e9b958e5abe39b5abc5b45ea0dcf05bb0861da9ede0081d48727a5c57b: Status 404 returned error can't find the container with id f68744e9b958e5abe39b5abc5b45ea0dcf05bb0861da9ede0081d48727a5c57b Jun 10 15:01:40.355671 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:40.355651 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49563c45_0b4c_404d_b5fd_46b64abfa6a8.slice/crio-5a08caa1242eaf91b13dc80d3ba7c7c158bb1de20ca22ed85662b3ce0fb80cd1 WatchSource:0}: Error finding container 5a08caa1242eaf91b13dc80d3ba7c7c158bb1de20ca22ed85662b3ce0fb80cd1: Status 404 returned error can't find the container with id 5a08caa1242eaf91b13dc80d3ba7c7c158bb1de20ca22ed85662b3ce0fb80cd1 Jun 10 15:01:40.357074 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:40.357012 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode836d9ae_14e4_4798_9250_669c90f1de68.slice/crio-755e55f76b6146032c9819f031df4a57e106d4d823bd808c2c56605e640bb9a6 WatchSource:0}: Error finding container 755e55f76b6146032c9819f031df4a57e106d4d823bd808c2c56605e640bb9a6: Status 404 returned error can't find the container with id 755e55f76b6146032c9819f031df4a57e106d4d823bd808c2c56605e640bb9a6 Jun 10 15:01:40.357906 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:40.357756 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d107589_1c68_4bf6_a05a_31045a73f204.slice/crio-b2df630c8d5c30cfa2ef9fa2cdd53c611644d4103408250768c3669a85a49017 WatchSource:0}: Error finding container b2df630c8d5c30cfa2ef9fa2cdd53c611644d4103408250768c3669a85a49017: Status 404 returned error can't find the container with id b2df630c8d5c30cfa2ef9fa2cdd53c611644d4103408250768c3669a85a49017 Jun 10 15:01:40.358602 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:01:40.358574 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91cc19fa_ee08_4fb7_bd80_84897bff8842.slice/crio-6e4d4b315cb85ddfcf53b0dbd86c957510d9a2df5e92fb48b44da15fcf0b1a1c WatchSource:0}: Error finding container 6e4d4b315cb85ddfcf53b0dbd86c957510d9a2df5e92fb48b44da15fcf0b1a1c: Status 404 returned error can't find the container with id 6e4d4b315cb85ddfcf53b0dbd86c957510d9a2df5e92fb48b44da15fcf0b1a1c Jun 10 15:01:40.992856 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:40.992781 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:40.992856 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:40.992846 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:40.993095 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:40.992973 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:40.993095 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:40.993030 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret podName:c4b47316-b098-43dc-aa43-973bca48c3c2 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:42.993013358 +0000 UTC m=+6.125306146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret") pod "global-pull-secret-syncer-fxmn5" (UID: "c4b47316-b098-43dc-aa43-973bca48c3c2") : object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:40.993480 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:40.993453 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:40.993576 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:40.993526 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:01:42.993503904 +0000 UTC m=+6.125796691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:41.099948 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.094501 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:41.103851 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:41.100664 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 15:01:41.103851 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:41.100689 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 15:01:41.103851 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:41.100702 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kzhdj for pod openshift-network-diagnostics/network-check-target-2mbrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:41.103851 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:41.100773 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj podName:07d53553-c4d6-4df1-a2f1-8d6a74ac5b72 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:43.100754791 +0000 UTC m=+6.233047569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzhdj" (UniqueName: "kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj") pod "network-check-target-2mbrv" (UID: "07d53553-c4d6-4df1-a2f1-8d6a74ac5b72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:41.320090 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.319986 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 14:56:39 +0000 UTC" deadline="2028-03-17 01:17:44.472091616 +0000 UTC" Jun 10 15:01:41.320090 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.320043 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15490h16m3.152052565s" Jun 10 15:01:41.340754 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.339906 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:41.340754 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:41.340064 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:41.340754 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.340444 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:41.340754 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:41.340532 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:41.340754 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.340593 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:41.340754 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:41.340664 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:41.365270 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.365243 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal" event={"ID":"bbfeb1994c2bac4a8dfa92f1e1823c15","Type":"ContainerStarted","Data":"3cba8d0e4135f3038efd72adde43442e115fa4e6461213166dfd3293f3b39cdf"} Jun 10 15:01:41.369522 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.369496 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c89rn" event={"ID":"91cc19fa-ee08-4fb7-bd80-84897bff8842","Type":"ContainerStarted","Data":"6e4d4b315cb85ddfcf53b0dbd86c957510d9a2df5e92fb48b44da15fcf0b1a1c"} Jun 10 15:01:41.373717 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.373692 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" event={"ID":"0d107589-1c68-4bf6-a05a-31045a73f204","Type":"ContainerStarted","Data":"b2df630c8d5c30cfa2ef9fa2cdd53c611644d4103408250768c3669a85a49017"} Jun 10 15:01:41.381241 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.381218 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-djm7n" event={"ID":"e512dd9b-89b7-4bb0-abef-1d717f999ddc","Type":"ContainerStarted","Data":"f68744e9b958e5abe39b5abc5b45ea0dcf05bb0861da9ede0081d48727a5c57b"} Jun 10 15:01:41.388082 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.388022 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-85.ec2.internal" podStartSLOduration=3.388005337 podStartE2EDuration="3.388005337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:01:41.387153653 +0000 UTC m=+4.519446451" watchObservedRunningTime="2026-06-10 15:01:41.388005337 +0000 UTC m=+4.520298132" Jun 10 15:01:41.405014 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.404989 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kvf47" event={"ID":"49563c45-0b4c-404d-b5fd-46b64abfa6a8","Type":"ContainerStarted","Data":"5a08caa1242eaf91b13dc80d3ba7c7c158bb1de20ca22ed85662b3ce0fb80cd1"} Jun 10 15:01:41.420612 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.420520 2561 generic.go:356] "Generic (PLEG): container finished" podID="c9a9f2b5539e7a75832f63deb43d54cb" containerID="d2d685ecc0852667ae22530f5792c04f443bc9f087fb10a71c04e9bb797e2852" exitCode=0 Jun 10 15:01:41.420612 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.420592 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" event={"ID":"c9a9f2b5539e7a75832f63deb43d54cb","Type":"ContainerDied","Data":"d2d685ecc0852667ae22530f5792c04f443bc9f087fb10a71c04e9bb797e2852"} Jun 10 15:01:41.444712 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.444690 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" event={"ID":"e836d9ae-14e4-4798-9250-669c90f1de68","Type":"ContainerStarted","Data":"755e55f76b6146032c9819f031df4a57e106d4d823bd808c2c56605e640bb9a6"} Jun 10 15:01:41.479312 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.479287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wqtc6" event={"ID":"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947","Type":"ContainerStarted","Data":"dc6d70bc7e9e0cd2ebc6b0062bf9a7e812bb2a51c45fb298a5a3d2d0196de06e"} Jun 10 15:01:41.486281 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:41.486255 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" event={"ID":"b73ecdfe-8356-4a0a-872b-1116ee1a95b9","Type":"ContainerStarted","Data":"68fb275715da64db310152d3edfdcfc4d526836886349853b7f4d01f8abdc844"} Jun 10 15:01:42.520207 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:42.520174 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" event={"ID":"c9a9f2b5539e7a75832f63deb43d54cb","Type":"ContainerStarted","Data":"5bd60a1f1bcbe04f070a66840be589de95c079ca67d460c2d9d003b179da6462"} Jun 10 15:01:43.013037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:43.012962 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:43.013175 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:43.013048 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:43.013175 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.013133 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:43.013175 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.013168 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:43.013336 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.013205 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:01:47.013183521 +0000 UTC m=+10.145476299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:43.013336 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.013227 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret podName:c4b47316-b098-43dc-aa43-973bca48c3c2 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:47.01321521 +0000 UTC m=+10.145507984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret") pod "global-pull-secret-syncer-fxmn5" (UID: "c4b47316-b098-43dc-aa43-973bca48c3c2") : object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:43.114650 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:43.114029 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:43.114650 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.114213 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 15:01:43.114650 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.114231 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 15:01:43.114650 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.114244 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kzhdj for pod openshift-network-diagnostics/network-check-target-2mbrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:43.114650 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.114297 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj podName:07d53553-c4d6-4df1-a2f1-8d6a74ac5b72 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:47.114277968 +0000 UTC m=+10.246570857 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzhdj" (UniqueName: "kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj") pod "network-check-target-2mbrv" (UID: "07d53553-c4d6-4df1-a2f1-8d6a74ac5b72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:43.338374 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:43.338298 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:43.338374 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:43.338338 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:43.338555 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.338469 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:43.338604 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.338571 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:43.338643 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:43.338621 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:43.338737 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:43.338717 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:45.337603 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:45.337564 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:45.338073 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:45.337688 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:45.338073 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:45.337564 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:45.338073 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:45.337790 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:45.338073 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:45.337564 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:45.338073 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:45.337887 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:47.047368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:47.047330 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:47.048018 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:47.047408 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:47.048018 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.047533 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:47.048018 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.047589 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:01:55.047570921 +0000 UTC m=+18.179863707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:47.048018 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.048009 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:47.048274 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.048060 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret podName:c4b47316-b098-43dc-aa43-973bca48c3c2 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:55.048045962 +0000 UTC m=+18.180338749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret") pod "global-pull-secret-syncer-fxmn5" (UID: "c4b47316-b098-43dc-aa43-973bca48c3c2") : object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:47.148287 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:47.148223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:47.148440 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.148371 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 15:01:47.148440 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.148392 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 15:01:47.148440 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.148404 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kzhdj for pod openshift-network-diagnostics/network-check-target-2mbrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:47.148606 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.148458 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj podName:07d53553-c4d6-4df1-a2f1-8d6a74ac5b72 nodeName:}" failed. No retries permitted until 2026-06-10 15:01:55.148441714 +0000 UTC m=+18.280734488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzhdj" (UniqueName: "kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj") pod "network-check-target-2mbrv" (UID: "07d53553-c4d6-4df1-a2f1-8d6a74ac5b72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:47.339426 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:47.338606 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:47.339426 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.338750 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:47.339426 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:47.339124 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:47.339426 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.339220 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:47.339426 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:47.339274 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:47.339426 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:47.339347 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:49.337317 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:49.337282 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:49.337754 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:49.337291 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:49.337754 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:49.337437 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:49.337754 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:49.337290 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:49.337754 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:49.337491 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:49.337754 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:49.337531 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:51.338676 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:51.338181 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:51.338676 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:51.338196 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:51.338676 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:51.338307 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:51.338676 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:51.338383 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:51.338676 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:51.338468 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:51.338676 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:51.338537 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:53.337673 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:53.337640 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:53.337673 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:53.337675 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:53.338192 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:53.337640 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:53.338192 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:53.337761 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:53.338192 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:53.337827 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:53.338192 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:53.337896 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:55.112474 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:55.112441 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:55.112914 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:55.112501 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:55.112914 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.112589 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:55.112914 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.112601 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:55.112914 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.112661 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:02:11.112639738 +0000 UTC m=+34.244932530 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:01:55.112914 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.112682 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret podName:c4b47316-b098-43dc-aa43-973bca48c3c2 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:11.112671731 +0000 UTC m=+34.244964508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret") pod "global-pull-secret-syncer-fxmn5" (UID: "c4b47316-b098-43dc-aa43-973bca48c3c2") : object "kube-system"/"original-pull-secret" not registered Jun 10 15:01:55.213791 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:55.213762 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:55.213959 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.213946 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 15:01:55.214021 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.213965 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 15:01:55.214021 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.213975 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kzhdj for pod openshift-network-diagnostics/network-check-target-2mbrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:55.214092 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.214036 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj podName:07d53553-c4d6-4df1-a2f1-8d6a74ac5b72 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:11.214018852 +0000 UTC m=+34.346311625 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzhdj" (UniqueName: "kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj") pod "network-check-target-2mbrv" (UID: "07d53553-c4d6-4df1-a2f1-8d6a74ac5b72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:01:55.337869 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:55.337840 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:55.338040 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:55.337845 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:55.338040 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.338011 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:55.338040 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:55.337847 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:55.338191 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.338085 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:55.338191 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:55.338163 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:57.338310 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:57.338092 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:57.338783 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:57.338161 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:57.338783 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:57.338423 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:57.338783 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:57.338185 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:57.338783 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:57.338496 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:57.338783 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:57.338555 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:59.337949 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.337822 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:01:59.338516 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:59.338027 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:01:59.338516 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.337823 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:01:59.338516 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:59.338095 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:01:59.338516 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.337823 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:01:59.338516 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:01:59.338138 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:01:59.551350 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.551319 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" event={"ID":"e836d9ae-14e4-4798-9250-669c90f1de68","Type":"ContainerStarted","Data":"6f6aa56199ccfe7dec43b586abbecc8f39d41219508213759fa0b83eae7500aa"} Jun 10 15:01:59.551449 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.551409 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" event={"ID":"e836d9ae-14e4-4798-9250-669c90f1de68","Type":"ContainerStarted","Data":"6bb432cdfe1f120c6146b190263797b2c359060d31c6caa8dcd9dec3ed3ebf33"} Jun 10 15:01:59.551505 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.551454 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" event={"ID":"e836d9ae-14e4-4798-9250-669c90f1de68","Type":"ContainerStarted","Data":"3df2151b636b2c655fc720bc3b9ba5893108835f094403b4b59e4fa8c83dbc36"} Jun 10 15:01:59.551547 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.551506 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" event={"ID":"e836d9ae-14e4-4798-9250-669c90f1de68","Type":"ContainerStarted","Data":"5cb519e6c02b61c1afbb06b743230248079ba91ed616b6f8b3332e2106c612c9"} Jun 10 15:01:59.551585 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.551556 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" event={"ID":"e836d9ae-14e4-4798-9250-669c90f1de68","Type":"ContainerStarted","Data":"2382cd99080236104aee2eb237db959056311b5375c4f27ff63a1263e68f95e4"} Jun 10 15:01:59.552698 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.552671 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wqtc6" event={"ID":"72f1e4b5-60f0-41a4-b9dd-2a83c4dfc947","Type":"ContainerStarted","Data":"32c5c4e2f6d8b296de101269acac6967a142affcdc211c69e82fc8d972939a12"} Jun 10 15:01:59.553958 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.553925 2561 generic.go:356] "Generic (PLEG): container finished" podID="b73ecdfe-8356-4a0a-872b-1116ee1a95b9" containerID="887026880ac98318161847aef7f322d9bb991030850dfc7d6d8572134b203b91" exitCode=0 Jun 10 15:01:59.554035 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.554002 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" event={"ID":"b73ecdfe-8356-4a0a-872b-1116ee1a95b9","Type":"ContainerDied","Data":"887026880ac98318161847aef7f322d9bb991030850dfc7d6d8572134b203b91"} Jun 10 15:01:59.555550 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.555527 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l575g" event={"ID":"53f26c7f-fc18-4658-b7a7-a29a2d392b25","Type":"ContainerStarted","Data":"1ba9bf778715901a5da2852372d94019dfd8885c07799d0006fbf4bcb556eeef"} Jun 10 15:01:59.556967 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.556944 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c89rn" event={"ID":"91cc19fa-ee08-4fb7-bd80-84897bff8842","Type":"ContainerStarted","Data":"0e2065ac1d8043baa02ce1f7d20174d27923144f5680d0fd509192eada48c2d4"} Jun 10 15:01:59.558644 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.558611 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" event={"ID":"0d107589-1c68-4bf6-a05a-31045a73f204","Type":"ContainerStarted","Data":"7bf4b9d90b1a6aac54f7a5af76b71f96c343be7add1d5b593570a521d9e23dcf"} Jun 10 15:01:59.560404 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.560154 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-djm7n" event={"ID":"e512dd9b-89b7-4bb0-abef-1d717f999ddc","Type":"ContainerStarted","Data":"5778479a3ecd3349ef18b7f58bbe0bfbf1e3a151c64881f112957821f6c7b4ae"} Jun 10 15:01:59.561442 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.561419 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kvf47" event={"ID":"49563c45-0b4c-404d-b5fd-46b64abfa6a8","Type":"ContainerStarted","Data":"ac7c33720793cd8ec424ce5d867cc3dcc76d6bfd60086f1821ecae875078555b"} Jun 10 15:01:59.572176 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.572134 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-85.ec2.internal" podStartSLOduration=21.572122904 podStartE2EDuration="21.572122904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:01:42.537675618 +0000 UTC m=+5.669968413" watchObservedRunningTime="2026-06-10 15:01:59.572122904 +0000 UTC m=+22.704415698" Jun 10 15:01:59.572280 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.572262 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wqtc6" podStartSLOduration=3.901212032 podStartE2EDuration="22.572258254s" totalImagesPullingTime="18.671046222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:01:59.571889065 +0000 UTC m=+22.704181860" watchObservedRunningTime="2026-06-10 15:01:59.572258254 +0000 UTC m=+22.704551058" Jun 10 15:01:59.590128 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.590088 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kvf47" podStartSLOduration=3.966467963 podStartE2EDuration="22.59007616s" totalImagesPullingTime="18.623608197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:01:59.58990921 +0000 UTC m=+22.722202016" watchObservedRunningTime="2026-06-10 15:01:59.59007616 +0000 UTC m=+22.722368954" Jun 10 15:01:59.606687 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.606652 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c89rn" podStartSLOduration=4.323326991 podStartE2EDuration="22.60663226s" totalImagesPullingTime="18.283305269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:01:59.606500374 +0000 UTC m=+22.738793167" watchObservedRunningTime="2026-06-10 15:01:59.60663226 +0000 UTC m=+22.738925054" Jun 10 15:01:59.628449 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.628413 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l575g" podStartSLOduration=3.992767695 podStartE2EDuration="22.628401273s" totalImagesPullingTime="18.635633578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:01:59.627896956 +0000 UTC m=+22.760189751" watchObservedRunningTime="2026-06-10 15:01:59.628401273 +0000 UTC m=+22.760694083" Jun 10 15:01:59.646089 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:01:59.646047 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-djm7n" podStartSLOduration=4.356915344 podStartE2EDuration="22.646030208s" totalImagesPullingTime="18.289114864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:01:59.645014468 +0000 UTC m=+22.777307266" watchObservedRunningTime="2026-06-10 15:01:59.646030208 +0000 UTC m=+22.778323002" Jun 10 15:02:00.061622 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.061602 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 10 15:02:00.329639 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.329321 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-10T15:02:00.061618109Z","UUID":"d4ab5116-23d6-41a5-8d10-ea804b41b9c8","Handler":null,"Name":"","Endpoint":""} Jun 10 15:02:00.330825 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.330802 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 10 15:02:00.330940 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.330847 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 10 15:02:00.453842 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.453814 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:02:00.454406 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.454384 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:02:00.565999 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.565968 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" event={"ID":"0d107589-1c68-4bf6-a05a-31045a73f204","Type":"ContainerStarted","Data":"d3a8ed37bbefc7d5a25ccd7488196a2335ab7861235eaf109bf7aa76606bfb61"} Jun 10 15:02:00.569373 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.569348 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" event={"ID":"e836d9ae-14e4-4798-9250-669c90f1de68","Type":"ContainerStarted","Data":"b442cf2b724c9fd3011a9b38c45dc40288a57ccbef9e490c6771bf496838ae90"} Jun 10 15:02:00.570094 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.569844 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:02:00.570337 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:00.570320 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c89rn" Jun 10 15:02:01.337883 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:01.337806 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:02:01.337883 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:01.337827 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:01.338108 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:01.337974 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:02:01.338108 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:01.338060 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:02:01.338219 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:01.338124 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:01.338219 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:01.338206 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:02:01.572915 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:01.572867 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nrp6w" event={"ID":"74e4cc5f-e66f-4eb2-84af-b6ce55aa4824","Type":"ContainerStarted","Data":"5e15e08147b5fcd15d534dd2d0b4218430df9069221d0b4799371ea8a5d9c288"} Jun 10 15:02:01.575412 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:01.575374 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" event={"ID":"0d107589-1c68-4bf6-a05a-31045a73f204","Type":"ContainerStarted","Data":"1beb53225d680718f1c7ac97868ecb8d5922150cad55b2667d6c6f050596b364"} Jun 10 15:02:01.591033 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:01.590955 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nrp6w" podStartSLOduration=5.936894185 podStartE2EDuration="24.590919526s" totalImagesPullingTime="18.654025341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:02:01.590607599 +0000 UTC m=+24.722900394" watchObservedRunningTime="2026-06-10 15:02:01.590919526 +0000 UTC m=+24.723212321" Jun 10 15:02:01.609826 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:01.609789 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rnrg9" podStartSLOduration=4.688785497 podStartE2EDuration="24.60977754s" totalImagesPullingTime="19.920992043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:02:01.609396331 +0000 UTC m=+24.741689125" watchObservedRunningTime="2026-06-10 15:02:01.60977754 +0000 UTC m=+24.742070366" Jun 10 15:02:02.580704 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:02.580660 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" event={"ID":"e836d9ae-14e4-4798-9250-669c90f1de68","Type":"ContainerStarted","Data":"aa6896b6bf44ec889e8a496e31e29851915b4f083721b06e720077cb0be37332"} Jun 10 15:02:03.337943 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:03.337557 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:02:03.337943 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:03.337600 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:03.337943 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:03.337562 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:03.337943 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:03.337698 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:02:03.337943 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:03.337781 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:02:03.337943 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:03.337876 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:02:04.585915 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:04.585742 2561 generic.go:356] "Generic (PLEG): container finished" podID="b73ecdfe-8356-4a0a-872b-1116ee1a95b9" containerID="75eae024f25cd06da1083f0a740946e5d18a2edd8966c53c5ed099a440bd91a3" exitCode=0 Jun 10 15:02:04.586700 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:04.585821 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" event={"ID":"b73ecdfe-8356-4a0a-872b-1116ee1a95b9","Type":"ContainerDied","Data":"75eae024f25cd06da1083f0a740946e5d18a2edd8966c53c5ed099a440bd91a3"} Jun 10 15:02:04.592854 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:04.592829 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" event={"ID":"e836d9ae-14e4-4798-9250-669c90f1de68","Type":"ContainerStarted","Data":"565f64e04e9ddcddf51a291233c62596e066227c55ceb123ce9dd255a69397d8"} Jun 10 15:02:04.593113 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:04.593092 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:02:04.593230 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:04.593217 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:02:04.593297 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:04.593283 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:02:04.607942 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:04.607910 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:02:04.608124 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:04.608109 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:02:04.643760 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:04.643725 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" podStartSLOduration=8.95201658 podStartE2EDuration="27.643713169s" totalImagesPullingTime="18.691696589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:02:04.642532045 +0000 UTC m=+27.774824838" watchObservedRunningTime="2026-06-10 15:02:04.643713169 +0000 UTC m=+27.776006003" Jun 10 15:02:05.338163 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:05.338131 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:05.338357 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:05.338131 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:02:05.338357 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:05.338235 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:02:05.338357 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:05.338267 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:05.338524 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:05.338390 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:02:05.338524 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:05.338486 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:02:06.000718 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:06.000661 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2mbrv"] Jun 10 15:02:06.001127 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:06.000760 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:06.001127 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:06.000846 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:02:06.002130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:06.002100 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fxmn5"] Jun 10 15:02:06.002242 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:06.002228 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:06.002348 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:06.002322 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:02:06.002922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:06.002902 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-msxf9"] Jun 10 15:02:06.003025 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:06.002997 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:02:06.003090 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:06.003074 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:02:06.598035 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:06.598006 2561 generic.go:356] "Generic (PLEG): container finished" podID="b73ecdfe-8356-4a0a-872b-1116ee1a95b9" containerID="458e5755b64034dfbf7df3edc101be86827b381c7bde0f560c7324deaf37f89d" exitCode=0 Jun 10 15:02:06.598192 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:06.598090 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" event={"ID":"b73ecdfe-8356-4a0a-872b-1116ee1a95b9","Type":"ContainerDied","Data":"458e5755b64034dfbf7df3edc101be86827b381c7bde0f560c7324deaf37f89d"} Jun 10 15:02:07.338422 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:07.338396 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:07.338776 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:07.338481 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:02:07.338776 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:07.338579 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:07.338776 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:07.338694 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:02:08.337922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:08.337778 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:02:08.338075 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:08.338032 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:02:08.603658 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:08.603598 2561 generic.go:356] "Generic (PLEG): container finished" podID="b73ecdfe-8356-4a0a-872b-1116ee1a95b9" containerID="2d37264bd601d999ecd41ba39a6b6ce32905bc1e461d96525b35d9375a0b2ea2" exitCode=0 Jun 10 15:02:08.603658 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:08.603636 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" event={"ID":"b73ecdfe-8356-4a0a-872b-1116ee1a95b9","Type":"ContainerDied","Data":"2d37264bd601d999ecd41ba39a6b6ce32905bc1e461d96525b35d9375a0b2ea2"} Jun 10 15:02:09.338131 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:09.338103 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:09.338291 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:09.338103 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:09.338291 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:09.338210 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:02:09.338291 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:09.338275 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:02:10.337180 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:10.337155 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:02:10.337518 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:10.337294 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:02:11.132137 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.132103 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:02:11.132306 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.132160 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:11.132306 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.132262 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 15:02:11.132396 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.132319 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret podName:c4b47316-b098-43dc-aa43-973bca48c3c2 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:43.132304624 +0000 UTC m=+66.264597396 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret") pod "global-pull-secret-syncer-fxmn5" (UID: "c4b47316-b098-43dc-aa43-973bca48c3c2") : object "kube-system"/"original-pull-secret" not registered Jun 10 15:02:11.132396 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.132262 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:02:11.132396 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.132376 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:02:43.132362947 +0000 UTC m=+66.264655732 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 15:02:11.233125 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.233090 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:11.233315 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.233286 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 15:02:11.233315 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.233317 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 15:02:11.233478 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.233330 2561 projected.go:196] Error preparing data for projected volume kube-api-access-kzhdj for pod openshift-network-diagnostics/network-check-target-2mbrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:02:11.233478 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.233387 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj podName:07d53553-c4d6-4df1-a2f1-8d6a74ac5b72 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:43.233368027 +0000 UTC m=+66.365660801 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kzhdj" (UniqueName: "kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj") pod "network-check-target-2mbrv" (UID: "07d53553-c4d6-4df1-a2f1-8d6a74ac5b72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 15:02:11.337452 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.337424 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:11.337813 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.337535 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2mbrv" podUID="07d53553-c4d6-4df1-a2f1-8d6a74ac5b72" Jun 10 15:02:11.337813 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.337624 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:11.337813 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.337793 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fxmn5" podUID="c4b47316-b098-43dc-aa43-973bca48c3c2" Jun 10 15:02:11.666685 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.666654 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-85.ec2.internal" event="NodeReady" Jun 10 15:02:11.666859 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.666802 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 10 15:02:11.708166 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.708135 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n"] Jun 10 15:02:11.716572 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.716551 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-ddf7f6777-mnb8x"] Jun 10 15:02:11.716724 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.716697 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:11.719228 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.719206 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 10 15:02:11.719228 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.719223 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pscgt\"" Jun 10 15:02:11.719386 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.719209 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 10 15:02:11.725557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.725538 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n"] Jun 10 15:02:11.725693 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.725680 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.727585 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.727567 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nh957"] Jun 10 15:02:11.728958 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.728904 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 10 15:02:11.728958 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.728904 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 10 15:02:11.728958 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.728950 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 10 15:02:11.729141 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.728998 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nnbbk\"" Jun 10 15:02:11.737952 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.737844 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ddf7f6777-mnb8x"] Jun 10 15:02:11.738096 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.738066 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.738257 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.738235 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cm2vf"] Jun 10 15:02:11.739460 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.739439 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 10 15:02:11.740638 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.740605 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 10 15:02:11.740727 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.740670 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-982f9\"" Jun 10 15:02:11.740797 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.740770 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 10 15:02:11.747171 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.747152 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nh957"] Jun 10 15:02:11.747307 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.747244 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cm2vf" Jun 10 15:02:11.748170 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.748152 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cm2vf"] Jun 10 15:02:11.749835 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.749757 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lvtpz\"" Jun 10 15:02:11.749835 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.749782 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 10 15:02:11.749835 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.749809 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 10 15:02:11.750047 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.749866 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 10 15:02:11.838963 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.838871 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff61f83-9ab8-403e-ad28-98afc53e8d00-ca-trust-extracted\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.838963 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.838907 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad836858-b9dc-4287-b3e8-907485054bf6-config-volume\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.838963 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.838955 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-bound-sa-token\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.839163 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.838996 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hdp7\" (UniqueName: \"kubernetes.io/projected/ad836858-b9dc-4287-b3e8-907485054bf6-kube-api-access-8hdp7\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.839163 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839060 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-image-registry-private-configuration\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.839163 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839100 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-installation-pull-secrets\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.839294 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839164 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-trusted-ca\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.839294 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839206 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.839294 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839231 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0531c81-3fe1-4f31-b45a-7aab9889966b-cert\") pod \"ingress-canary-cm2vf\" (UID: \"a0531c81-3fe1-4f31-b45a-7aab9889966b\") " pod="openshift-ingress-canary/ingress-canary-cm2vf" Jun 10 15:02:11.839438 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839300 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-certificates\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.839438 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839338 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:11.839438 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839371 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad836858-b9dc-4287-b3e8-907485054bf6-tmp-dir\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.839438 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839406 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thkv5\" (UniqueName: \"kubernetes.io/projected/a0531c81-3fe1-4f31-b45a-7aab9889966b-kube-api-access-thkv5\") pod \"ingress-canary-cm2vf\" (UID: \"a0531c81-3fe1-4f31-b45a-7aab9889966b\") " pod="openshift-ingress-canary/ingress-canary-cm2vf" Jun 10 15:02:11.839588 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839447 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/75ab486a-c093-4fb3-bc99-0174f164ad3d-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:11.839588 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839482 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn844\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-kube-api-access-sn844\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.839588 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.839497 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.940767 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.940743 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff61f83-9ab8-403e-ad28-98afc53e8d00-ca-trust-extracted\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.940767 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.940769 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad836858-b9dc-4287-b3e8-907485054bf6-config-volume\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.941130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.940786 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-bound-sa-token\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.941130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.940807 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hdp7\" (UniqueName: \"kubernetes.io/projected/ad836858-b9dc-4287-b3e8-907485054bf6-kube-api-access-8hdp7\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.941130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941024 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-image-registry-private-configuration\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.941130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941059 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-installation-pull-secrets\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.941130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941128 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-trusted-ca\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.941331 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941156 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.941331 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941170 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff61f83-9ab8-403e-ad28-98afc53e8d00-ca-trust-extracted\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.941331 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941178 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0531c81-3fe1-4f31-b45a-7aab9889966b-cert\") pod \"ingress-canary-cm2vf\" (UID: \"a0531c81-3fe1-4f31-b45a-7aab9889966b\") " pod="openshift-ingress-canary/ingress-canary-cm2vf" Jun 10 15:02:11.941331 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941238 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-certificates\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.941331 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.941251 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 15:02:11.941331 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.941266 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddf7f6777-mnb8x: secret "image-registry-tls" not found Jun 10 15:02:11.941331 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.941326 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 15:02:11.941593 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941386 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad836858-b9dc-4287-b3e8-907485054bf6-config-volume\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.941800 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941264 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:11.941800 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.941331 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls podName:aff61f83-9ab8-403e-ad28-98afc53e8d00 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:12.441313051 +0000 UTC m=+35.573605839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls") pod "image-registry-ddf7f6777-mnb8x" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00") : secret "image-registry-tls" not found Jun 10 15:02:11.941956 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.941836 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert podName:75ab486a-c093-4fb3-bc99-0174f164ad3d nodeName:}" failed. No retries permitted until 2026-06-10 15:02:12.441817132 +0000 UTC m=+35.574109917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-c8w7n" (UID: "75ab486a-c093-4fb3-bc99-0174f164ad3d") : secret "networking-console-plugin-cert" not found Jun 10 15:02:11.941956 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941905 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-certificates\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.941956 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941950 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad836858-b9dc-4287-b3e8-907485054bf6-tmp-dir\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.942093 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.941989 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thkv5\" (UniqueName: \"kubernetes.io/projected/a0531c81-3fe1-4f31-b45a-7aab9889966b-kube-api-access-thkv5\") pod \"ingress-canary-cm2vf\" (UID: \"a0531c81-3fe1-4f31-b45a-7aab9889966b\") " pod="openshift-ingress-canary/ingress-canary-cm2vf" Jun 10 15:02:11.942093 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.942015 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/75ab486a-c093-4fb3-bc99-0174f164ad3d-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:11.942093 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.942043 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn844\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-kube-api-access-sn844\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.942208 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.942093 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.942208 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.942199 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 15:02:11.942208 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.942202 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-trusted-ca\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.942324 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.942208 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad836858-b9dc-4287-b3e8-907485054bf6-tmp-dir\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.942324 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:11.942250 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls podName:ad836858-b9dc-4287-b3e8-907485054bf6 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:12.442234556 +0000 UTC m=+35.574527342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls") pod "dns-default-nh957" (UID: "ad836858-b9dc-4287-b3e8-907485054bf6") : secret "dns-default-metrics-tls" not found Jun 10 15:02:11.942708 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.942662 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/75ab486a-c093-4fb3-bc99-0174f164ad3d-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:11.945898 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.945840 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-installation-pull-secrets\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.945898 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.945872 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-image-registry-private-configuration\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.946066 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.945901 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0531c81-3fe1-4f31-b45a-7aab9889966b-cert\") pod \"ingress-canary-cm2vf\" (UID: \"a0531c81-3fe1-4f31-b45a-7aab9889966b\") " pod="openshift-ingress-canary/ingress-canary-cm2vf" Jun 10 15:02:11.953471 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.953423 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-bound-sa-token\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.953725 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.953698 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hdp7\" (UniqueName: \"kubernetes.io/projected/ad836858-b9dc-4287-b3e8-907485054bf6-kube-api-access-8hdp7\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:11.955414 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.955393 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn844\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-kube-api-access-sn844\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:11.955507 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:11.955425 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thkv5\" (UniqueName: \"kubernetes.io/projected/a0531c81-3fe1-4f31-b45a-7aab9889966b-kube-api-access-thkv5\") pod \"ingress-canary-cm2vf\" (UID: \"a0531c81-3fe1-4f31-b45a-7aab9889966b\") " pod="openshift-ingress-canary/ingress-canary-cm2vf" Jun 10 15:02:12.057325 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.057302 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cm2vf" Jun 10 15:02:12.100122 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.099871 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:12.214024 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.213997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:12.337723 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.337697 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:02:12.340750 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.340728 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 15:02:12.340750 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.340743 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sz4k2\"" Jun 10 15:02:12.446200 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.446145 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:12.446200 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.446195 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:12.446371 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.446219 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:12.446371 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:12.446321 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 15:02:12.446371 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:12.446331 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 15:02:12.446371 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:12.446372 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert podName:75ab486a-c093-4fb3-bc99-0174f164ad3d nodeName:}" failed. No retries permitted until 2026-06-10 15:02:13.446356788 +0000 UTC m=+36.578649562 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-c8w7n" (UID: "75ab486a-c093-4fb3-bc99-0174f164ad3d") : secret "networking-console-plugin-cert" not found Jun 10 15:02:12.446581 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:12.446389 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 15:02:12.446581 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:12.446426 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddf7f6777-mnb8x: secret "image-registry-tls" not found Jun 10 15:02:12.446581 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:12.446389 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls podName:ad836858-b9dc-4287-b3e8-907485054bf6 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:13.446381031 +0000 UTC m=+36.578673803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls") pod "dns-default-nh957" (UID: "ad836858-b9dc-4287-b3e8-907485054bf6") : secret "dns-default-metrics-tls" not found Jun 10 15:02:12.446581 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:12.446468 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls podName:aff61f83-9ab8-403e-ad28-98afc53e8d00 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:13.446453254 +0000 UTC m=+36.578746028 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls") pod "image-registry-ddf7f6777-mnb8x" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00") : secret "image-registry-tls" not found Jun 10 15:02:12.615004 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:12.614979 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cm2vf"] Jun 10 15:02:12.618676 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:02:12.618655 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0531c81_3fe1_4f31_b45a_7aab9889966b.slice/crio-7ad9611f917381b5577e89298f6945b62864f0b17d19b682f5a024bd108758ec WatchSource:0}: Error finding container 7ad9611f917381b5577e89298f6945b62864f0b17d19b682f5a024bd108758ec: Status 404 returned error can't find the container with id 7ad9611f917381b5577e89298f6945b62864f0b17d19b682f5a024bd108758ec Jun 10 15:02:13.337412 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.337388 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:13.337557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.337542 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:13.340562 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.340544 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 15:02:13.340562 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.340553 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 15:02:13.341278 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.341263 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 15:02:13.341341 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.341265 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8lwdb\"" Jun 10 15:02:13.453185 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.453158 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:13.453293 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.453218 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:13.453293 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.453252 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:13.453386 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:13.453349 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 15:02:13.453433 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:13.453352 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 15:02:13.453433 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:13.453414 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert podName:75ab486a-c093-4fb3-bc99-0174f164ad3d nodeName:}" failed. No retries permitted until 2026-06-10 15:02:15.453395809 +0000 UTC m=+38.585688593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-c8w7n" (UID: "75ab486a-c093-4fb3-bc99-0174f164ad3d") : secret "networking-console-plugin-cert" not found Jun 10 15:02:13.453552 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:13.453473 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls podName:ad836858-b9dc-4287-b3e8-907485054bf6 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:15.453454744 +0000 UTC m=+38.585747529 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls") pod "dns-default-nh957" (UID: "ad836858-b9dc-4287-b3e8-907485054bf6") : secret "dns-default-metrics-tls" not found Jun 10 15:02:13.453552 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:13.453403 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 15:02:13.453552 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:13.453493 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddf7f6777-mnb8x: secret "image-registry-tls" not found Jun 10 15:02:13.453552 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:13.453532 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls podName:aff61f83-9ab8-403e-ad28-98afc53e8d00 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:15.453521029 +0000 UTC m=+38.585813800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls") pod "image-registry-ddf7f6777-mnb8x" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00") : secret "image-registry-tls" not found Jun 10 15:02:13.617022 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.616965 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cm2vf" event={"ID":"a0531c81-3fe1-4f31-b45a-7aab9889966b","Type":"ContainerStarted","Data":"7ad9611f917381b5577e89298f6945b62864f0b17d19b682f5a024bd108758ec"} Jun 10 15:02:13.620542 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.620519 2561 generic.go:356] "Generic (PLEG): container finished" podID="b73ecdfe-8356-4a0a-872b-1116ee1a95b9" containerID="840c319750f10a9e7e5a9cb89305382472e9fbd8cf89656f7ea415df0d6fba64" exitCode=0 Jun 10 15:02:13.620632 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:13.620553 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" event={"ID":"b73ecdfe-8356-4a0a-872b-1116ee1a95b9","Type":"ContainerDied","Data":"840c319750f10a9e7e5a9cb89305382472e9fbd8cf89656f7ea415df0d6fba64"} Jun 10 15:02:14.627212 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:14.627175 2561 generic.go:356] "Generic (PLEG): container finished" podID="b73ecdfe-8356-4a0a-872b-1116ee1a95b9" containerID="b08535d05155d3b1b518beef326c2d34942da4de72f35f57aada5c36a650cc78" exitCode=0 Jun 10 15:02:14.627212 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:14.627207 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" event={"ID":"b73ecdfe-8356-4a0a-872b-1116ee1a95b9","Type":"ContainerDied","Data":"b08535d05155d3b1b518beef326c2d34942da4de72f35f57aada5c36a650cc78"} Jun 10 15:02:15.467777 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:15.467625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:15.467777 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:15.467770 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 15:02:15.467982 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:15.467795 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:15.467982 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:15.467821 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert podName:75ab486a-c093-4fb3-bc99-0174f164ad3d nodeName:}" failed. No retries permitted until 2026-06-10 15:02:19.467807354 +0000 UTC m=+42.600100125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-c8w7n" (UID: "75ab486a-c093-4fb3-bc99-0174f164ad3d") : secret "networking-console-plugin-cert" not found Jun 10 15:02:15.467982 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:15.467866 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 15:02:15.467982 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:15.467882 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:15.467982 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:15.467903 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls podName:ad836858-b9dc-4287-b3e8-907485054bf6 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:19.467892481 +0000 UTC m=+42.600185253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls") pod "dns-default-nh957" (UID: "ad836858-b9dc-4287-b3e8-907485054bf6") : secret "dns-default-metrics-tls" not found Jun 10 15:02:15.467982 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:15.467947 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 15:02:15.467982 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:15.467958 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddf7f6777-mnb8x: secret "image-registry-tls" not found Jun 10 15:02:15.468213 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:15.467990 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls podName:aff61f83-9ab8-403e-ad28-98afc53e8d00 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:19.467982444 +0000 UTC m=+42.600275216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls") pod "image-registry-ddf7f6777-mnb8x" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00") : secret "image-registry-tls" not found Jun 10 15:02:15.630458 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:15.630435 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cm2vf" event={"ID":"a0531c81-3fe1-4f31-b45a-7aab9889966b","Type":"ContainerStarted","Data":"8178e821ccd29bb466e518d2824ecb6fd575a00986269d6ba08c1160d8c34e91"} Jun 10 15:02:15.633443 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:15.633412 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" event={"ID":"b73ecdfe-8356-4a0a-872b-1116ee1a95b9","Type":"ContainerStarted","Data":"2dd210c1a7aa18867ac82668e0129c096de487e7c4068f96cbcafe83a7324283"} Jun 10 15:02:15.650671 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:15.650634 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cm2vf" podStartSLOduration=2.308959824 podStartE2EDuration="4.650615829s" totalImagesPullingTime="2.341656005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:02:15.650330316 +0000 UTC m=+38.782623110" watchObservedRunningTime="2026-06-10 15:02:15.650615829 +0000 UTC m=+38.782908613" Jun 10 15:02:15.677386 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:15.677328 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v8gqd" podStartSLOduration=9.600504287 podStartE2EDuration="38.677316089s" totalImagesPullingTime="29.076811802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:02:15.676057451 +0000 UTC m=+38.808350255" watchObservedRunningTime="2026-06-10 15:02:15.677316089 +0000 UTC m=+38.809608880" Jun 10 15:02:15.845216 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:15.845195 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:15.943672 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:15.943628 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:16.020477 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:16.020456 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:16.111262 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:16.111240 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:16.315712 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:16.315690 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:16.416564 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:16.416540 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:17.548606 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:17.548580 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:17.612939 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:17.612900 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:18.091262 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:18.091240 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:18.191978 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:18.191959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:19.496092 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:19.496067 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:19.496438 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:19.496116 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:19.496438 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:19.496151 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:19.496438 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:19.496203 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 15:02:19.496438 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:19.496216 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 15:02:19.496438 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:19.496268 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert podName:75ab486a-c093-4fb3-bc99-0174f164ad3d nodeName:}" failed. No retries permitted until 2026-06-10 15:02:27.496253686 +0000 UTC m=+50.628546457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-c8w7n" (UID: "75ab486a-c093-4fb3-bc99-0174f164ad3d") : secret "networking-console-plugin-cert" not found Jun 10 15:02:19.496438 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:19.496271 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 15:02:19.496438 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:19.496219 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddf7f6777-mnb8x: secret "image-registry-tls" not found Jun 10 15:02:19.496438 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:19.496318 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls podName:ad836858-b9dc-4287-b3e8-907485054bf6 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:27.496302045 +0000 UTC m=+50.628594816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls") pod "dns-default-nh957" (UID: "ad836858-b9dc-4287-b3e8-907485054bf6") : secret "dns-default-metrics-tls" not found Jun 10 15:02:19.496438 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:19.496356 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls podName:aff61f83-9ab8-403e-ad28-98afc53e8d00 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:27.496342205 +0000 UTC m=+50.628634976 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls") pod "image-registry-ddf7f6777-mnb8x" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00") : secret "image-registry-tls" not found Jun 10 15:02:25.548442 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:25.548413 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:25.653800 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:25.653770 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:26.176406 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.176380 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72"] Jun 10 15:02:26.189899 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.189868 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.190817 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.190791 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72"] Jun 10 15:02:26.192283 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.192262 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 10 15:02:26.192375 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.192299 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 10 15:02:26.192375 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.192332 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 10 15:02:26.193267 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.193250 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 10 15:02:26.241156 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.241129 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/163ce46a-ece4-42b3-8d69-ba8a9d03ff22-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bbf87b647-s2b72\" (UID: \"163ce46a-ece4-42b3-8d69-ba8a9d03ff22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.241236 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.241177 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/163ce46a-ece4-42b3-8d69-ba8a9d03ff22-tmp\") pod \"klusterlet-addon-workmgr-6bbf87b647-s2b72\" (UID: \"163ce46a-ece4-42b3-8d69-ba8a9d03ff22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.241236 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.241203 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b265k\" (UniqueName: \"kubernetes.io/projected/163ce46a-ece4-42b3-8d69-ba8a9d03ff22-kube-api-access-b265k\") pod \"klusterlet-addon-workmgr-6bbf87b647-s2b72\" (UID: \"163ce46a-ece4-42b3-8d69-ba8a9d03ff22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.341909 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.341885 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/163ce46a-ece4-42b3-8d69-ba8a9d03ff22-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bbf87b647-s2b72\" (UID: \"163ce46a-ece4-42b3-8d69-ba8a9d03ff22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.342016 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.341966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/163ce46a-ece4-42b3-8d69-ba8a9d03ff22-tmp\") pod \"klusterlet-addon-workmgr-6bbf87b647-s2b72\" (UID: \"163ce46a-ece4-42b3-8d69-ba8a9d03ff22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.342016 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.341995 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b265k\" (UniqueName: \"kubernetes.io/projected/163ce46a-ece4-42b3-8d69-ba8a9d03ff22-kube-api-access-b265k\") pod \"klusterlet-addon-workmgr-6bbf87b647-s2b72\" (UID: \"163ce46a-ece4-42b3-8d69-ba8a9d03ff22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.342355 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.342335 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/163ce46a-ece4-42b3-8d69-ba8a9d03ff22-tmp\") pod \"klusterlet-addon-workmgr-6bbf87b647-s2b72\" (UID: \"163ce46a-ece4-42b3-8d69-ba8a9d03ff22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.346016 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.345994 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/163ce46a-ece4-42b3-8d69-ba8a9d03ff22-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bbf87b647-s2b72\" (UID: \"163ce46a-ece4-42b3-8d69-ba8a9d03ff22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.351750 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.351730 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b265k\" (UniqueName: \"kubernetes.io/projected/163ce46a-ece4-42b3-8d69-ba8a9d03ff22-kube-api-access-b265k\") pod \"klusterlet-addon-workmgr-6bbf87b647-s2b72\" (UID: \"163ce46a-ece4-42b3-8d69-ba8a9d03ff22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.499692 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.499641 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:26.631707 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.631679 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72"] Jun 10 15:02:26.635790 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:02:26.635768 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163ce46a_ece4_42b3_8d69_ba8a9d03ff22.slice/crio-27bd6d917ef27ff44ab39414ef46780a8159e2be0a658a4d73311893c8e1aa30 WatchSource:0}: Error finding container 27bd6d917ef27ff44ab39414ef46780a8159e2be0a658a4d73311893c8e1aa30: Status 404 returned error can't find the container with id 27bd6d917ef27ff44ab39414ef46780a8159e2be0a658a4d73311893c8e1aa30 Jun 10 15:02:26.657412 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:26.657387 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" event={"ID":"163ce46a-ece4-42b3-8d69-ba8a9d03ff22","Type":"ContainerStarted","Data":"27bd6d917ef27ff44ab39414ef46780a8159e2be0a658a4d73311893c8e1aa30"} Jun 10 15:02:27.170012 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:27.169989 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:27.246445 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:27.246416 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:27.550199 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:27.550180 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:27.550301 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:27.550217 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:27.550301 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:27.550246 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:27.550411 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:27.550325 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 15:02:27.550411 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:27.550340 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddf7f6777-mnb8x: secret "image-registry-tls" not found Jun 10 15:02:27.550411 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:27.550340 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 15:02:27.550411 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:27.550347 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 15:02:27.550597 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:27.550394 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls podName:aff61f83-9ab8-403e-ad28-98afc53e8d00 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:43.550380846 +0000 UTC m=+66.682673620 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls") pod "image-registry-ddf7f6777-mnb8x" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00") : secret "image-registry-tls" not found Jun 10 15:02:27.550597 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:27.550438 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert podName:75ab486a-c093-4fb3-bc99-0174f164ad3d nodeName:}" failed. No retries permitted until 2026-06-10 15:02:43.550420388 +0000 UTC m=+66.682713161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-c8w7n" (UID: "75ab486a-c093-4fb3-bc99-0174f164ad3d") : secret "networking-console-plugin-cert" not found Jun 10 15:02:27.550597 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:27.550461 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls podName:ad836858-b9dc-4287-b3e8-907485054bf6 nodeName:}" failed. No retries permitted until 2026-06-10 15:02:43.550451647 +0000 UTC m=+66.682744420 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls") pod "dns-default-nh957" (UID: "ad836858-b9dc-4287-b3e8-907485054bf6") : secret "dns-default-metrics-tls" not found Jun 10 15:02:27.728821 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:27.728803 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:27.799733 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:27.799709 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:32.674499 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:32.674464 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" event={"ID":"163ce46a-ece4-42b3-8d69-ba8a9d03ff22","Type":"ContainerStarted","Data":"b37bed4d715bdaafac2fdc66843be006636b28573910ddbb44d31fbad555f322"} Jun 10 15:02:32.674871 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:32.674680 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:32.676224 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:32.676207 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:02:32.697159 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:32.697120 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" podStartSLOduration=1.40481104 podStartE2EDuration="6.697108052s" totalImagesPullingTime="5.292297012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:02:32.696557887 +0000 UTC m=+55.828850680" watchObservedRunningTime="2026-06-10 15:02:32.697108052 +0000 UTC m=+55.829400845" Jun 10 15:02:36.614525 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:36.614498 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pkch" Jun 10 15:02:36.790675 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:36.790648 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:36.867550 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:36.867492 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:38.551034 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:38.551002 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:38.627523 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:38.627497 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:43.159994 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.159957 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:02:43.160364 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.160019 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:43.162701 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.162684 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 15:02:43.162829 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.162814 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 15:02:43.170982 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:43.170963 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 15:02:43.171055 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:43.171043 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:03:47.171022389 +0000 UTC m=+130.303315160 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : secret "metrics-daemon-secret" not found Jun 10 15:02:43.172993 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.172975 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4b47316-b098-43dc-aa43-973bca48c3c2-original-pull-secret\") pod \"global-pull-secret-syncer-fxmn5\" (UID: \"c4b47316-b098-43dc-aa43-973bca48c3c2\") " pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:43.260447 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.260424 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:43.263065 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.263049 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 15:02:43.273532 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.273518 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 15:02:43.283606 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.283584 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzhdj\" (UniqueName: \"kubernetes.io/projected/07d53553-c4d6-4df1-a2f1-8d6a74ac5b72-kube-api-access-kzhdj\") pod \"network-check-target-2mbrv\" (UID: \"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72\") " pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:43.347378 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.347358 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fxmn5" Jun 10 15:02:43.355062 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.355041 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8lwdb\"" Jun 10 15:02:43.363262 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.363241 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:43.472114 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.472088 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fxmn5"] Jun 10 15:02:43.474911 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:02:43.474888 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b47316_b098_43dc_aa43_973bca48c3c2.slice/crio-9a92678b4efa4fef5568ec5da2f1c0c4713248ba1917baa7be291730caa1b53c WatchSource:0}: Error finding container 9a92678b4efa4fef5568ec5da2f1c0c4713248ba1917baa7be291730caa1b53c: Status 404 returned error can't find the container with id 9a92678b4efa4fef5568ec5da2f1c0c4713248ba1917baa7be291730caa1b53c Jun 10 15:02:43.487997 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.487975 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2mbrv"] Jun 10 15:02:43.490587 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:02:43.490564 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07d53553_c4d6_4df1_a2f1_8d6a74ac5b72.slice/crio-067827ae122ffadc26d9c0be97a265435c9c4af7fa50e53b3d8a35bee54cb2c8 WatchSource:0}: Error finding container 067827ae122ffadc26d9c0be97a265435c9c4af7fa50e53b3d8a35bee54cb2c8: Status 404 returned error can't find the container with id 067827ae122ffadc26d9c0be97a265435c9c4af7fa50e53b3d8a35bee54cb2c8 Jun 10 15:02:43.563633 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.563597 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:02:43.563736 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.563649 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:02:43.563736 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.563694 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:02:43.563837 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:43.563762 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 15:02:43.563837 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:43.563780 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddf7f6777-mnb8x: secret "image-registry-tls" not found Jun 10 15:02:43.563837 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:43.563792 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 15:02:43.563837 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:43.563797 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 15:02:43.564039 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:43.563838 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls podName:aff61f83-9ab8-403e-ad28-98afc53e8d00 nodeName:}" failed. No retries permitted until 2026-06-10 15:03:15.563816148 +0000 UTC m=+98.696108936 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls") pod "image-registry-ddf7f6777-mnb8x" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00") : secret "image-registry-tls" not found Jun 10 15:02:43.564039 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:43.563857 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls podName:ad836858-b9dc-4287-b3e8-907485054bf6 nodeName:}" failed. No retries permitted until 2026-06-10 15:03:15.563847499 +0000 UTC m=+98.696140279 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls") pod "dns-default-nh957" (UID: "ad836858-b9dc-4287-b3e8-907485054bf6") : secret "dns-default-metrics-tls" not found Jun 10 15:02:43.564039 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:02:43.563873 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert podName:75ab486a-c093-4fb3-bc99-0174f164ad3d nodeName:}" failed. No retries permitted until 2026-06-10 15:03:15.563864301 +0000 UTC m=+98.696157077 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-c8w7n" (UID: "75ab486a-c093-4fb3-bc99-0174f164ad3d") : secret "networking-console-plugin-cert" not found Jun 10 15:02:43.700096 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.700036 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2mbrv" event={"ID":"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72","Type":"ContainerStarted","Data":"067827ae122ffadc26d9c0be97a265435c9c4af7fa50e53b3d8a35bee54cb2c8"} Jun 10 15:02:43.701108 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:43.701092 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fxmn5" event={"ID":"c4b47316-b098-43dc-aa43-973bca48c3c2","Type":"ContainerStarted","Data":"9a92678b4efa4fef5568ec5da2f1c0c4713248ba1917baa7be291730caa1b53c"} Jun 10 15:02:46.711239 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:46.711138 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2mbrv" event={"ID":"07d53553-c4d6-4df1-a2f1-8d6a74ac5b72","Type":"ContainerStarted","Data":"5a29f5aef0030f6d92f52d3073c2e625bdbf7530a973aa28e82ff7b837322e48"} Jun 10 15:02:46.711659 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:46.711249 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:02:46.730600 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:46.730558 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2mbrv" podStartSLOduration=66.841972981 podStartE2EDuration="1m9.730545644s" totalImagesPullingTime="2.888572663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:02:46.729785212 +0000 UTC m=+69.862078004" watchObservedRunningTime="2026-06-10 15:02:46.730545644 +0000 UTC m=+69.862838436" Jun 10 15:02:48.716854 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:48.716820 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fxmn5" event={"ID":"c4b47316-b098-43dc-aa43-973bca48c3c2","Type":"ContainerStarted","Data":"4f6672582b5a56b7a0c150c7431bff2fb451e1d44edda8094e296f0527ab0c8d"} Jun 10 15:02:48.736578 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:48.736539 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fxmn5" podStartSLOduration=67.385580373 podStartE2EDuration="1m11.736528319s" totalImagesPullingTime="4.350947946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:02:48.736234238 +0000 UTC m=+71.868527032" watchObservedRunningTime="2026-06-10 15:02:48.736528319 +0000 UTC m=+71.868821112" Jun 10 15:02:49.148700 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:49.148679 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:49.219490 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:49.219470 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:52.025458 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:52.025431 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:02:52.123344 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:02:52.123320 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:03:15.573978 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:03:15.573916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:03:15.574459 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:03:15.573987 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:03:15.574459 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:03:15.574014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:03:15.574459 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:03:15.574085 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 15:03:15.574459 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:03:15.574107 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 15:03:15.574459 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:03:15.574120 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 15:03:15.574459 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:03:15.574159 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls podName:ad836858-b9dc-4287-b3e8-907485054bf6 nodeName:}" failed. No retries permitted until 2026-06-10 15:04:19.574145605 +0000 UTC m=+162.706438376 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls") pod "dns-default-nh957" (UID: "ad836858-b9dc-4287-b3e8-907485054bf6") : secret "dns-default-metrics-tls" not found Jun 10 15:03:15.574459 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:03:15.574182 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert podName:75ab486a-c093-4fb3-bc99-0174f164ad3d nodeName:}" failed. No retries permitted until 2026-06-10 15:04:19.574168549 +0000 UTC m=+162.706461321 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-c8w7n" (UID: "75ab486a-c093-4fb3-bc99-0174f164ad3d") : secret "networking-console-plugin-cert" not found Jun 10 15:03:15.574459 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:03:15.574108 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddf7f6777-mnb8x: secret "image-registry-tls" not found Jun 10 15:03:15.574459 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:03:15.574287 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls podName:aff61f83-9ab8-403e-ad28-98afc53e8d00 nodeName:}" failed. No retries permitted until 2026-06-10 15:04:19.574271282 +0000 UTC m=+162.706564062 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls") pod "image-registry-ddf7f6777-mnb8x" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00") : secret "image-registry-tls" not found Jun 10 15:03:17.715949 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:03:17.715903 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2mbrv" Jun 10 15:03:18.474607 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:03:18.474560 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:03:18.548354 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:03:18.548314 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:03:47.213292 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:03:47.213236 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:03:47.213767 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:03:47.213388 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 15:03:47.213767 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:03:47.213459 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:05:49.213443458 +0000 UTC m=+252.345736231 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : secret "metrics-daemon-secret" not found Jun 10 15:04:14.728119 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:14.728079 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" podUID="75ab486a-c093-4fb3-bc99-0174f164ad3d" Jun 10 15:04:14.739212 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:14.739188 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" podUID="aff61f83-9ab8-403e-ad28-98afc53e8d00" Jun 10 15:04:14.748464 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:14.748447 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nh957" podUID="ad836858-b9dc-4287-b3e8-907485054bf6" Jun 10 15:04:14.917369 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:14.917344 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:04:14.917475 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:14.917344 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:04:14.917528 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:14.917344 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nh957" Jun 10 15:04:15.348189 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:15.348110 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:04:19.645682 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:19.645643 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:04:19.646134 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:19.645722 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:04:19.646134 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:19.645764 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:04:19.646134 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:19.645785 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 15:04:19.646134 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:19.645844 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls podName:ad836858-b9dc-4287-b3e8-907485054bf6 nodeName:}" failed. No retries permitted until 2026-06-10 15:06:21.645829128 +0000 UTC m=+284.778121900 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls") pod "dns-default-nh957" (UID: "ad836858-b9dc-4287-b3e8-907485054bf6") : secret "dns-default-metrics-tls" not found Jun 10 15:04:19.646134 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:19.645867 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 15:04:19.646134 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:19.645869 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 15:04:19.646134 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:19.645889 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ddf7f6777-mnb8x: secret "image-registry-tls" not found Jun 10 15:04:19.646134 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:19.645959 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert podName:75ab486a-c093-4fb3-bc99-0174f164ad3d nodeName:}" failed. No retries permitted until 2026-06-10 15:06:21.64592095 +0000 UTC m=+284.778213730 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-c8w7n" (UID: "75ab486a-c093-4fb3-bc99-0174f164ad3d") : secret "networking-console-plugin-cert" not found Jun 10 15:04:19.646134 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:04:19.645982 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls podName:aff61f83-9ab8-403e-ad28-98afc53e8d00 nodeName:}" failed. No retries permitted until 2026-06-10 15:06:21.645969314 +0000 UTC m=+284.778262092 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls") pod "image-registry-ddf7f6777-mnb8x" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00") : secret "image-registry-tls" not found Jun 10 15:04:30.338177 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:30.338141 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:04:32.675556 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:32.675519 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" podUID="163ce46a-ece4-42b3-8d69-ba8a9d03ff22" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.10:8000/readyz\": dial tcp 10.132.0.10:8000: connect: connection refused" Jun 10 15:04:32.961018 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:32.960952 2561 generic.go:356] "Generic (PLEG): container finished" podID="163ce46a-ece4-42b3-8d69-ba8a9d03ff22" containerID="b37bed4d715bdaafac2fdc66843be006636b28573910ddbb44d31fbad555f322" exitCode=1 Jun 10 15:04:32.961126 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:32.961023 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" event={"ID":"163ce46a-ece4-42b3-8d69-ba8a9d03ff22","Type":"ContainerDied","Data":"b37bed4d715bdaafac2fdc66843be006636b28573910ddbb44d31fbad555f322"} Jun 10 15:04:32.961358 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:32.961336 2561 scope.go:117] "RemoveContainer" containerID="b37bed4d715bdaafac2fdc66843be006636b28573910ddbb44d31fbad555f322" Jun 10 15:04:33.964877 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:33.964842 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" event={"ID":"163ce46a-ece4-42b3-8d69-ba8a9d03ff22","Type":"ContainerStarted","Data":"d5b623939a6df115b5a752467a2f02c579f883674c4327d44a1acbae88552efd"} Jun 10 15:04:33.965250 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:33.965115 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:04:33.965761 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:04:33.965737 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:05:09.741552 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:09.741519 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:18.335473 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:18.335441 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:37.601342 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.601307 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r"] Jun 10 15:05:37.603569 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.603549 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r" Jun 10 15:05:37.607126 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.607108 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bqkfb\"" Jun 10 15:05:37.621805 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.621778 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r"] Jun 10 15:05:37.705940 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.705915 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6c55898649-2nnlx"] Jun 10 15:05:37.707639 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.707625 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf"] Jun 10 15:05:37.707753 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.707739 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.709494 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.709475 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf" Jun 10 15:05:37.712914 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.712894 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 10 15:05:37.713136 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.713120 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 10 15:05:37.713216 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.713200 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 10 15:05:37.713359 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.713338 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 10 15:05:37.713359 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.713342 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 10 15:05:37.713943 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.713915 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 10 15:05:37.714182 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.714167 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wnqz9\"" Jun 10 15:05:37.714826 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.714811 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 10 15:05:37.718185 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.718170 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 10 15:05:37.723053 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.723038 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5"] Jun 10 15:05:37.725466 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.725450 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf"] Jun 10 15:05:37.725543 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.725513 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:37.726404 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.726389 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vwr46\"" Jun 10 15:05:37.730146 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.730121 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 10 15:05:37.730255 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.730172 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 10 15:05:37.730324 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.730128 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 10 15:05:37.730401 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.730379 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-74zjs\"" Jun 10 15:05:37.736547 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.736530 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 10 15:05:37.753846 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.753820 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5"] Jun 10 15:05:37.762902 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.762872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5kq2\" (UniqueName: \"kubernetes.io/projected/011d70d7-83ea-4b2e-8df9-2960564861bd-kube-api-access-k5kq2\") pod \"network-check-source-dbd9cf844-x9p2r\" (UID: \"011d70d7-83ea-4b2e-8df9-2960564861bd\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r" Jun 10 15:05:37.816750 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.816724 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c55898649-2nnlx"] Jun 10 15:05:37.863299 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863246 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-stats-auth\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.863299 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863270 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5qpg\" (UniqueName: \"kubernetes.io/projected/53a0dcee-a209-4ffb-8f8c-a30209a86b19-kube-api-access-k5qpg\") pod \"volume-data-source-validator-78dd696465-jrfwf\" (UID: \"53a0dcee-a209-4ffb-8f8c-a30209a86b19\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf" Jun 10 15:05:37.863299 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863289 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h4bh\" (UniqueName: \"kubernetes.io/projected/516ba81a-80cd-49fe-bf46-f1eb09dae80b-kube-api-access-6h4bh\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:37.863443 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863352 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxjdd\" (UniqueName: \"kubernetes.io/projected/10bad15f-1514-446e-980c-33b2a8b5ca02-kube-api-access-sxjdd\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.863443 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863390 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5kq2\" (UniqueName: \"kubernetes.io/projected/011d70d7-83ea-4b2e-8df9-2960564861bd-kube-api-access-k5kq2\") pod \"network-check-source-dbd9cf844-x9p2r\" (UID: \"011d70d7-83ea-4b2e-8df9-2960564861bd\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r" Jun 10 15:05:37.863443 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863414 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:37.863536 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863460 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/516ba81a-80cd-49fe-bf46-f1eb09dae80b-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:37.863536 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863481 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-default-certificate\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.863536 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863510 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.863627 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.863561 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.879106 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.879082 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5kq2\" (UniqueName: \"kubernetes.io/projected/011d70d7-83ea-4b2e-8df9-2960564861bd-kube-api-access-k5kq2\") pod \"network-check-source-dbd9cf844-x9p2r\" (UID: \"011d70d7-83ea-4b2e-8df9-2960564861bd\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r" Jun 10 15:05:37.911859 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.911839 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r" Jun 10 15:05:37.964659 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.964634 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxjdd\" (UniqueName: \"kubernetes.io/projected/10bad15f-1514-446e-980c-33b2a8b5ca02-kube-api-access-sxjdd\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.964775 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.964680 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:37.964775 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.964728 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/516ba81a-80cd-49fe-bf46-f1eb09dae80b-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:37.964775 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.964747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-default-certificate\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.964918 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.964778 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.964918 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.964827 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.964918 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.964868 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-stats-auth\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.964918 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.964894 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5qpg\" (UniqueName: \"kubernetes.io/projected/53a0dcee-a209-4ffb-8f8c-a30209a86b19-kube-api-access-k5qpg\") pod \"volume-data-source-validator-78dd696465-jrfwf\" (UID: \"53a0dcee-a209-4ffb-8f8c-a30209a86b19\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf" Jun 10 15:05:37.964918 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.964916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h4bh\" (UniqueName: \"kubernetes.io/projected/516ba81a-80cd-49fe-bf46-f1eb09dae80b-kube-api-access-6h4bh\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:37.965190 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:37.965149 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 15:05:37.965244 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:37.965239 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:38.465217856 +0000 UTC m=+241.597510630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : secret "router-metrics-certs-default" not found Jun 10 15:05:37.965729 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.965525 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/516ba81a-80cd-49fe-bf46-f1eb09dae80b-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:37.965729 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:37.965585 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:37.965729 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:37.965654 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls podName:516ba81a-80cd-49fe-bf46-f1eb09dae80b nodeName:}" failed. No retries permitted until 2026-06-10 15:05:38.465638073 +0000 UTC m=+241.597930863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-d67p5" (UID: "516ba81a-80cd-49fe-bf46-f1eb09dae80b") : secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:37.965729 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:37.965720 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:38.46567154 +0000 UTC m=+241.597964315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : configmap references non-existent config key: service-ca.crt Jun 10 15:05:37.968197 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.968171 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-stats-auth\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.968821 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.968782 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-default-certificate\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.977564 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.977544 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxjdd\" (UniqueName: \"kubernetes.io/projected/10bad15f-1514-446e-980c-33b2a8b5ca02-kube-api-access-sxjdd\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:37.979642 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.979599 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h4bh\" (UniqueName: \"kubernetes.io/projected/516ba81a-80cd-49fe-bf46-f1eb09dae80b-kube-api-access-6h4bh\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:37.979846 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:37.979829 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5qpg\" (UniqueName: \"kubernetes.io/projected/53a0dcee-a209-4ffb-8f8c-a30209a86b19-kube-api-access-k5qpg\") pod \"volume-data-source-validator-78dd696465-jrfwf\" (UID: \"53a0dcee-a209-4ffb-8f8c-a30209a86b19\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf" Jun 10 15:05:38.021629 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.021609 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf" Jun 10 15:05:38.040191 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.040172 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r"] Jun 10 15:05:38.042831 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:05:38.042807 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011d70d7_83ea_4b2e_8df9_2960564861bd.slice/crio-bc0db9104d321e532cccc118fba3b5b488ead123ed6a94bc175000781cfa3cb1 WatchSource:0}: Error finding container bc0db9104d321e532cccc118fba3b5b488ead123ed6a94bc175000781cfa3cb1: Status 404 returned error can't find the container with id bc0db9104d321e532cccc118fba3b5b488ead123ed6a94bc175000781cfa3cb1 Jun 10 15:05:38.047143 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.047122 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:38.120175 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.120104 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r" event={"ID":"011d70d7-83ea-4b2e-8df9-2960564861bd","Type":"ContainerStarted","Data":"9588caaeb998f2030c4a07b6ebd81e8456c2e54438e1280adddec64f781a97aa"} Jun 10 15:05:38.120403 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.120382 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r" event={"ID":"011d70d7-83ea-4b2e-8df9-2960564861bd","Type":"ContainerStarted","Data":"bc0db9104d321e532cccc118fba3b5b488ead123ed6a94bc175000781cfa3cb1"} Jun 10 15:05:38.158204 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.158178 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:38.161598 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.161559 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-x9p2r" podStartSLOduration=1.161546488 podStartE2EDuration="1.161546488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:05:38.160557117 +0000 UTC m=+241.292849910" watchObservedRunningTime="2026-06-10 15:05:38.161546488 +0000 UTC m=+241.293839282" Jun 10 15:05:38.168627 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.168604 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf"] Jun 10 15:05:38.172824 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:05:38.172793 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a0dcee_a209_4ffb_8f8c_a30209a86b19.slice/crio-21557dbbedcae06f48441a6b3b37d4eca11dc51cc17b721bf8be6329aecb2c10 WatchSource:0}: Error finding container 21557dbbedcae06f48441a6b3b37d4eca11dc51cc17b721bf8be6329aecb2c10: Status 404 returned error can't find the container with id 21557dbbedcae06f48441a6b3b37d4eca11dc51cc17b721bf8be6329aecb2c10 Jun 10 15:05:38.468706 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.468658 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:38.468706 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.468702 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:38.468815 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:38.468796 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 15:05:38.468815 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:38.468797 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:38.468896 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:38.468846 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:39.468834539 +0000 UTC m=+242.601127312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : secret "router-metrics-certs-default" not found Jun 10 15:05:38.468965 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:38.468921 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:38.469005 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:38.468978 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:39.468961403 +0000 UTC m=+242.601254189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : configmap references non-existent config key: service-ca.crt Jun 10 15:05:38.469005 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:38.468996 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls podName:516ba81a-80cd-49fe-bf46-f1eb09dae80b nodeName:}" failed. No retries permitted until 2026-06-10 15:05:39.468990415 +0000 UTC m=+242.601283186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-d67p5" (UID: "516ba81a-80cd-49fe-bf46-f1eb09dae80b") : secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:39.124762 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:39.124730 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf" event={"ID":"53a0dcee-a209-4ffb-8f8c-a30209a86b19","Type":"ContainerStarted","Data":"21557dbbedcae06f48441a6b3b37d4eca11dc51cc17b721bf8be6329aecb2c10"} Jun 10 15:05:39.477185 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:39.476995 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:39.477185 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:39.477063 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:39.477185 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:39.477133 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:41.477110309 +0000 UTC m=+244.609403099 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : configmap references non-existent config key: service-ca.crt Jun 10 15:05:39.477444 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:39.477198 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 15:05:39.477444 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:39.477199 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:39.477444 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:39.477246 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:41.477229843 +0000 UTC m=+244.609522646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : secret "router-metrics-certs-default" not found Jun 10 15:05:39.477444 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:39.477326 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:39.477444 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:39.477381 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls podName:516ba81a-80cd-49fe-bf46-f1eb09dae80b nodeName:}" failed. No retries permitted until 2026-06-10 15:05:41.477366388 +0000 UTC m=+244.609659168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-d67p5" (UID: "516ba81a-80cd-49fe-bf46-f1eb09dae80b") : secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:40.128206 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:40.128174 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf" event={"ID":"53a0dcee-a209-4ffb-8f8c-a30209a86b19","Type":"ContainerStarted","Data":"62adccca96df3a87c9ba1c9e50714961b7e806d3066ef3d93597d2f9980da5af"} Jun 10 15:05:40.146519 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:40.146466 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-jrfwf" podStartSLOduration=1.858583825 podStartE2EDuration="3.146450512s" totalImagesPullingTime="1.287866687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:05:40.146053021 +0000 UTC m=+243.278345814" watchObservedRunningTime="2026-06-10 15:05:40.146450512 +0000 UTC m=+243.278743283" Jun 10 15:05:40.250037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:40.250017 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:41.059205 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:41.059181 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:41.170615 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:41.170591 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:41.490871 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:41.490778 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:41.490871 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:41.490855 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:41.491074 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:41.490895 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:41.491074 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:41.490909 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 15:05:41.491074 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:41.490987 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:45.490972936 +0000 UTC m=+248.623265708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : secret "router-metrics-certs-default" not found Jun 10 15:05:41.491074 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:41.491013 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:41.491074 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:41.491023 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:45.491007251 +0000 UTC m=+248.623300023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : configmap references non-existent config key: service-ca.crt Jun 10 15:05:41.491074 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:41.491064 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls podName:516ba81a-80cd-49fe-bf46-f1eb09dae80b nodeName:}" failed. No retries permitted until 2026-06-10 15:05:45.491054655 +0000 UTC m=+248.623347431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-d67p5" (UID: "516ba81a-80cd-49fe-bf46-f1eb09dae80b") : secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:42.434546 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:42.434506 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:42.583636 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:42.583606 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:42.679749 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:42.679723 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:42.784258 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:42.784197 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:42.970807 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:42.970785 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-djm7n_e512dd9b-89b7-4bb0-abef-1d717f999ddc/dns-node-resolver/0.log" Jun 10 15:05:43.182208 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:43.182181 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:43.345413 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:43.345392 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:43.969382 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:43.969358 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kvf47_49563c45-0b4c-404d-b5fd-46b64abfa6a8/node-ca/0.log" Jun 10 15:05:44.577714 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.577690 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cm2vf_a0531c81-3fe1-4f31-b45a-7aab9889966b/serve-healthcheck-canary/0.log" Jun 10 15:05:44.673036 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.673010 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:44.806676 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.806643 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-ct577"] Jun 10 15:05:44.809161 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.809143 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.810343 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.810320 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0386ee1b-67fd-4120-af37-960d7466bc9a-signing-cabundle\") pod \"service-ca-7d74d578b8-ct577\" (UID: \"0386ee1b-67fd-4120-af37-960d7466bc9a\") " pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.810441 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.810371 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99v27\" (UniqueName: \"kubernetes.io/projected/0386ee1b-67fd-4120-af37-960d7466bc9a-kube-api-access-99v27\") pod \"service-ca-7d74d578b8-ct577\" (UID: \"0386ee1b-67fd-4120-af37-960d7466bc9a\") " pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.810441 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.810403 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0386ee1b-67fd-4120-af37-960d7466bc9a-signing-key\") pod \"service-ca-7d74d578b8-ct577\" (UID: \"0386ee1b-67fd-4120-af37-960d7466bc9a\") " pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.818012 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.817995 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 10 15:05:44.818189 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.818172 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 10 15:05:44.818302 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.818286 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 10 15:05:44.818899 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.818884 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 10 15:05:44.830725 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.830676 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4dp4d\"" Jun 10 15:05:44.850581 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.850559 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-ct577"] Jun 10 15:05:44.894912 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.894885 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:44.911005 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.910983 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99v27\" (UniqueName: \"kubernetes.io/projected/0386ee1b-67fd-4120-af37-960d7466bc9a-kube-api-access-99v27\") pod \"service-ca-7d74d578b8-ct577\" (UID: \"0386ee1b-67fd-4120-af37-960d7466bc9a\") " pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.911071 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.911025 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0386ee1b-67fd-4120-af37-960d7466bc9a-signing-key\") pod \"service-ca-7d74d578b8-ct577\" (UID: \"0386ee1b-67fd-4120-af37-960d7466bc9a\") " pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.911203 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.911187 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0386ee1b-67fd-4120-af37-960d7466bc9a-signing-cabundle\") pod \"service-ca-7d74d578b8-ct577\" (UID: \"0386ee1b-67fd-4120-af37-960d7466bc9a\") " pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.911717 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.911693 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0386ee1b-67fd-4120-af37-960d7466bc9a-signing-cabundle\") pod \"service-ca-7d74d578b8-ct577\" (UID: \"0386ee1b-67fd-4120-af37-960d7466bc9a\") " pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.913227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.913207 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0386ee1b-67fd-4120-af37-960d7466bc9a-signing-key\") pod \"service-ca-7d74d578b8-ct577\" (UID: \"0386ee1b-67fd-4120-af37-960d7466bc9a\") " pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.921964 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.921922 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99v27\" (UniqueName: \"kubernetes.io/projected/0386ee1b-67fd-4120-af37-960d7466bc9a-kube-api-access-99v27\") pod \"service-ca-7d74d578b8-ct577\" (UID: \"0386ee1b-67fd-4120-af37-960d7466bc9a\") " pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:44.995829 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:44.995779 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:45.117295 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:45.117247 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-ct577" Jun 10 15:05:45.255780 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:45.255755 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-ct577"] Jun 10 15:05:45.259570 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:05:45.259542 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0386ee1b_67fd_4120_af37_960d7466bc9a.slice/crio-808a37c6c04a004b0720819956e17ba47df99510e863c8a6bc53668f3ed18d01 WatchSource:0}: Error finding container 808a37c6c04a004b0720819956e17ba47df99510e863c8a6bc53668f3ed18d01: Status 404 returned error can't find the container with id 808a37c6c04a004b0720819956e17ba47df99510e863c8a6bc53668f3ed18d01 Jun 10 15:05:45.514175 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:45.514091 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:45.514175 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:45.514135 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:45.514346 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:45.514180 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:45.514346 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:45.514266 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:45.514346 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:45.514269 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:53.514247739 +0000 UTC m=+256.646540518 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : configmap references non-existent config key: service-ca.crt Jun 10 15:05:45.514346 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:45.514294 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 15:05:45.514346 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:45.514302 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls podName:516ba81a-80cd-49fe-bf46-f1eb09dae80b nodeName:}" failed. No retries permitted until 2026-06-10 15:05:53.514291335 +0000 UTC m=+256.646584107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-d67p5" (UID: "516ba81a-80cd-49fe-bf46-f1eb09dae80b") : secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:45.514608 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:45.514355 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:05:53.514344293 +0000 UTC m=+256.646637066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : secret "router-metrics-certs-default" not found Jun 10 15:05:46.146639 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:46.146600 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-ct577" event={"ID":"0386ee1b-67fd-4120-af37-960d7466bc9a","Type":"ContainerStarted","Data":"808a37c6c04a004b0720819956e17ba47df99510e863c8a6bc53668f3ed18d01"} Jun 10 15:05:48.138068 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:48.138041 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:48.153460 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:48.153432 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-ct577" event={"ID":"0386ee1b-67fd-4120-af37-960d7466bc9a","Type":"ContainerStarted","Data":"737e0355bf32fd8b29225bf020192b1a130b157846edd879f5d6c7a30b55b4f2"} Jun 10 15:05:48.186369 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:48.186294 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-ct577" podStartSLOduration=1.531906104 podStartE2EDuration="4.186280813s" totalImagesPullingTime="2.654374709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:05:48.185238763 +0000 UTC m=+251.317531556" watchObservedRunningTime="2026-06-10 15:05:48.186280813 +0000 UTC m=+251.318573611" Jun 10 15:05:49.240158 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:49.240120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:05:49.240579 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:49.240270 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 15:05:49.240579 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:49.240335 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs podName:6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce nodeName:}" failed. No retries permitted until 2026-06-10 15:07:51.24031687 +0000 UTC m=+374.372609642 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs") pod "network-metrics-daemon-msxf9" (UID: "6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce") : secret "metrics-daemon-secret" not found Jun 10 15:05:49.618345 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:49.618310 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:49.808806 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:49.808774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:49.909993 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:49.909904 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:50.017114 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:50.017084 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:50.120331 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:50.120301 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:50.344582 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:50.344554 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:50.440394 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:50.440369 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:50.934675 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:50.934650 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:51.513069 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:51.513040 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:52.105806 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:52.105778 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:52.318508 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:52.318480 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:52.710465 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:52.710428 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:52.923726 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:52.923695 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:53.306881 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:53.306844 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:53.515126 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:53.515095 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:53.574550 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:53.574484 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:53.574550 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:53.574547 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:05:53.574747 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:53.574595 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:05:53.574747 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:53.574650 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 15:05:53.574747 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:53.574703 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:53.574747 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:53.574714 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:06:09.574698365 +0000 UTC m=+272.706991151 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : configmap references non-existent config key: service-ca.crt Jun 10 15:05:53.574747 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:53.574733 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs podName:10bad15f-1514-446e-980c-33b2a8b5ca02 nodeName:}" failed. No retries permitted until 2026-06-10 15:06:09.574724383 +0000 UTC m=+272.707017161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs") pod "router-default-6c55898649-2nnlx" (UID: "10bad15f-1514-446e-980c-33b2a8b5ca02") : secret "router-metrics-certs-default" not found Jun 10 15:05:53.574747 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:05:53.574748 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls podName:516ba81a-80cd-49fe-bf46-f1eb09dae80b nodeName:}" failed. No retries permitted until 2026-06-10 15:06:09.574739174 +0000 UTC m=+272.707031953 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-d67p5" (UID: "516ba81a-80cd-49fe-bf46-f1eb09dae80b") : secret "cluster-monitoring-operator-tls" not found Jun 10 15:05:54.036361 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:54.036302 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:54.136544 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:54.136518 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:54.327712 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:54.327690 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:54.715922 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:54.715863 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:54.915106 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:54.915061 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:55.533184 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:55.533152 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:55.766633 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:55.766606 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:56.098022 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:56.097978 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:56.208061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:56.208039 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:56.312505 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:56.312482 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:56.504446 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:56.504361 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:56.907807 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:56.907777 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:57.304037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:57.303997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:57.855504 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:57.855473 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:57.959993 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:57.959963 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:58.301785 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:58.301755 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:58.702664 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:58.702605 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:59.096169 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:59.096147 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:59.300022 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:59.299998 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:05:59.893896 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:05:59.893871 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:00.102169 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:00.102144 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:00.300265 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:00.300243 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:02.692817 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:02.692786 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:02.799552 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:02.799520 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:02.865772 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:02.865751 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:02.998996 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:02.998924 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:03.164185 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:03.164163 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:03.229440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:03.229416 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:03.374896 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:03.374876 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:03.452153 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:03.452132 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:03.578795 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:03.578776 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:03.779903 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:03.779855 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:04.184376 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:04.184352 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:04.388145 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:04.388113 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:04.608248 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:04.608225 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:05.210812 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:05.210787 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:05.293721 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:05.293698 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:09.590170 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.590130 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:06:09.590632 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.590184 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:06:09.590632 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.590218 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:06:09.590771 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.590753 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10bad15f-1514-446e-980c-33b2a8b5ca02-service-ca-bundle\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:06:09.592596 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.592572 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/516ba81a-80cd-49fe-bf46-f1eb09dae80b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-d67p5\" (UID: \"516ba81a-80cd-49fe-bf46-f1eb09dae80b\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:06:09.592701 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.592572 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bad15f-1514-446e-980c-33b2a8b5ca02-metrics-certs\") pod \"router-default-6c55898649-2nnlx\" (UID: \"10bad15f-1514-446e-980c-33b2a8b5ca02\") " pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:06:09.712686 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.712658 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-bjzrv"] Jun 10 15:06:09.715311 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.715298 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-bjzrv" Jun 10 15:06:09.717477 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.717461 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gt689\"" Jun 10 15:06:09.717783 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.717768 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 10 15:06:09.718197 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.718180 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 10 15:06:09.726187 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.726169 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-bjzrv"] Jun 10 15:06:09.791661 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.791641 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsx99\" (UniqueName: \"kubernetes.io/projected/0f451555-80a8-4b4b-a876-9b49611345f6-kube-api-access-vsx99\") pod \"downloads-6657875f97-bjzrv\" (UID: \"0f451555-80a8-4b4b-a876-9b49611345f6\") " pod="openshift-console/downloads-6657875f97-bjzrv" Jun 10 15:06:09.816604 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.816581 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:06:09.833668 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.833647 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" Jun 10 15:06:09.836115 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.836092 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vws5z"] Jun 10 15:06:09.838943 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.838912 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.841327 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.841269 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 10 15:06:09.841555 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.841536 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 10 15:06:09.841993 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.841972 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 10 15:06:09.842215 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.842192 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zpmwr\"" Jun 10 15:06:09.842463 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.842436 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 10 15:06:09.851749 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.851730 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vws5z"] Jun 10 15:06:09.855556 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.855534 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:09.892795 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.892729 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsx99\" (UniqueName: \"kubernetes.io/projected/0f451555-80a8-4b4b-a876-9b49611345f6-kube-api-access-vsx99\") pod \"downloads-6657875f97-bjzrv\" (UID: \"0f451555-80a8-4b4b-a876-9b49611345f6\") " pod="openshift-console/downloads-6657875f97-bjzrv" Jun 10 15:06:09.892923 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.892819 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvswj\" (UniqueName: \"kubernetes.io/projected/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-kube-api-access-wvswj\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.892923 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.892872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.893394 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.892952 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-data-volume\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.893394 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.893096 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.893394 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.893137 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-crio-socket\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.903190 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.903167 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsx99\" (UniqueName: \"kubernetes.io/projected/0f451555-80a8-4b4b-a876-9b49611345f6-kube-api-access-vsx99\") pod \"downloads-6657875f97-bjzrv\" (UID: \"0f451555-80a8-4b4b-a876-9b49611345f6\") " pod="openshift-console/downloads-6657875f97-bjzrv" Jun 10 15:06:09.966793 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.966768 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c55898649-2nnlx"] Jun 10 15:06:09.969692 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:09.969669 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bad15f_1514_446e_980c_33b2a8b5ca02.slice/crio-efb1a9f67eb050ae3f7ce8acc5c20920db150ec46a520b989163a88ae0dce323 WatchSource:0}: Error finding container efb1a9f67eb050ae3f7ce8acc5c20920db150ec46a520b989163a88ae0dce323: Status 404 returned error can't find the container with id efb1a9f67eb050ae3f7ce8acc5c20920db150ec46a520b989163a88ae0dce323 Jun 10 15:06:09.978222 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.978205 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5"] Jun 10 15:06:09.980292 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:09.980265 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod516ba81a_80cd_49fe_bf46_f1eb09dae80b.slice/crio-660b526d9a50ec92001db83a03fc0749bd19cb0090881f4cf59763d216c982b3 WatchSource:0}: Error finding container 660b526d9a50ec92001db83a03fc0749bd19cb0090881f4cf59763d216c982b3: Status 404 returned error can't find the container with id 660b526d9a50ec92001db83a03fc0749bd19cb0090881f4cf59763d216c982b3 Jun 10 15:06:09.985211 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.985187 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:09.994313 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.994295 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvswj\" (UniqueName: \"kubernetes.io/projected/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-kube-api-access-wvswj\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.994406 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.994329 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.994406 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.994354 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-data-volume\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.994546 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.994524 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.994610 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.994560 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-crio-socket\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.994667 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.994653 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-data-volume\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.994722 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.994671 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-crio-socket\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.995003 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.994988 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:09.996497 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:09.996477 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:10.004798 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.004772 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvswj\" (UniqueName: \"kubernetes.io/projected/012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4-kube-api-access-wvswj\") pod \"insights-runtime-extractor-vws5z\" (UID: \"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4\") " pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:10.025129 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.025067 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-bjzrv" Jun 10 15:06:10.090119 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.090088 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:10.156082 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.156051 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vws5z" Jun 10 15:06:10.156482 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.156463 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-bjzrv"] Jun 10 15:06:10.159397 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:10.159373 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f451555_80a8_4b4b_a876_9b49611345f6.slice/crio-447efb5f1aaec86491dbeb947bece555c5f44e2910bd8d9c633c922d775ff9ac WatchSource:0}: Error finding container 447efb5f1aaec86491dbeb947bece555c5f44e2910bd8d9c633c922d775ff9ac: Status 404 returned error can't find the container with id 447efb5f1aaec86491dbeb947bece555c5f44e2910bd8d9c633c922d775ff9ac Jun 10 15:06:10.213863 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.213827 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" event={"ID":"516ba81a-80cd-49fe-bf46-f1eb09dae80b","Type":"ContainerStarted","Data":"660b526d9a50ec92001db83a03fc0749bd19cb0090881f4cf59763d216c982b3"} Jun 10 15:06:10.214981 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.214956 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-bjzrv" event={"ID":"0f451555-80a8-4b4b-a876-9b49611345f6","Type":"ContainerStarted","Data":"447efb5f1aaec86491dbeb947bece555c5f44e2910bd8d9c633c922d775ff9ac"} Jun 10 15:06:10.216569 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.216539 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c55898649-2nnlx" event={"ID":"10bad15f-1514-446e-980c-33b2a8b5ca02","Type":"ContainerStarted","Data":"ce06306365b173fe1c79f5779d5794ecc164e7f9f989edf1a621308502fdc975"} Jun 10 15:06:10.216664 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.216600 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c55898649-2nnlx" event={"ID":"10bad15f-1514-446e-980c-33b2a8b5ca02","Type":"ContainerStarted","Data":"efb1a9f67eb050ae3f7ce8acc5c20920db150ec46a520b989163a88ae0dce323"} Jun 10 15:06:10.322908 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.322821 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6c55898649-2nnlx" podStartSLOduration=33.322805389 podStartE2EDuration="33.322805389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:10.245418559 +0000 UTC m=+273.377711352" watchObservedRunningTime="2026-06-10 15:06:10.322805389 +0000 UTC m=+273.455098183" Jun 10 15:06:10.323100 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.323085 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vws5z"] Jun 10 15:06:10.325127 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:10.325103 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012ed0c3_d0cf_4704_a8f6_7d6d5f83a2c4.slice/crio-d67799738d1c733ed53d3d50cdbfd840ea204c151580a03bc5f948b6735049e9 WatchSource:0}: Error finding container d67799738d1c733ed53d3d50cdbfd840ea204c151580a03bc5f948b6735049e9: Status 404 returned error can't find the container with id d67799738d1c733ed53d3d50cdbfd840ea204c151580a03bc5f948b6735049e9 Jun 10 15:06:10.817503 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.817463 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:06:10.819897 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:10.819873 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:06:11.223072 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:11.223038 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vws5z" event={"ID":"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4","Type":"ContainerStarted","Data":"959c0f33caa9b9b2ab29f1868628dc4d37bb2c50e6c213fddca135db073334b5"} Jun 10 15:06:11.223231 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:11.223109 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vws5z" event={"ID":"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4","Type":"ContainerStarted","Data":"adfbca1fd0e7b6393fc226a0b7942f5800a41844fb1d7a440a9287090214adb8"} Jun 10 15:06:11.223231 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:11.223140 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vws5z" event={"ID":"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4","Type":"ContainerStarted","Data":"d67799738d1c733ed53d3d50cdbfd840ea204c151580a03bc5f948b6735049e9"} Jun 10 15:06:11.223530 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:11.223505 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:06:11.224711 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:11.224691 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6c55898649-2nnlx" Jun 10 15:06:11.238816 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:11.238786 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:11.334128 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:11.334026 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:13.162229 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.162186 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:13.233424 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.233374 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" event={"ID":"516ba81a-80cd-49fe-bf46-f1eb09dae80b","Type":"ContainerStarted","Data":"ef4c37c9a86337c6f34a238041c7adcfabdd8f83b5147ad46d6534109cbdcd2c"} Jun 10 15:06:13.235754 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.235726 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vws5z" event={"ID":"012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4","Type":"ContainerStarted","Data":"01838a86a9115943f53c9d6ad83b0e90c793f251ba5943f24dafcbd5fb72f9ce"} Jun 10 15:06:13.255329 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.255276 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-d67p5" podStartSLOduration=33.519307861 podStartE2EDuration="36.255262158s" totalImagesPullingTime="2.735954297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:13.25459657 +0000 UTC m=+276.386889352" watchObservedRunningTime="2026-06-10 15:06:13.255262158 +0000 UTC m=+276.387554957" Jun 10 15:06:13.263404 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.263381 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:13.276140 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.276105 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vws5z" podStartSLOduration=1.995132054 podStartE2EDuration="4.276089104s" totalImagesPullingTime="2.28095705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:06:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:13.275444737 +0000 UTC m=+276.407737532" watchObservedRunningTime="2026-06-10 15:06:13.276089104 +0000 UTC m=+276.408381899" Jun 10 15:06:13.364458 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.364370 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:13.364968 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.364948 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl"] Jun 10 15:06:13.367335 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.367307 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" Jun 10 15:06:13.369711 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.369692 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 10 15:06:13.369830 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.369696 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-nvbnf\"" Jun 10 15:06:13.378225 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.378206 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl"] Jun 10 15:06:13.424591 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.424530 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0f56f66-1cbf-40e8-bea8-edaa9473f4c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-tvwzl\" (UID: \"b0f56f66-1cbf-40e8-bea8-edaa9473f4c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" Jun 10 15:06:13.518516 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.518492 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:13.525582 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.525554 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0f56f66-1cbf-40e8-bea8-edaa9473f4c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-tvwzl\" (UID: \"b0f56f66-1cbf-40e8-bea8-edaa9473f4c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" Jun 10 15:06:13.528233 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.528215 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0f56f66-1cbf-40e8-bea8-edaa9473f4c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-tvwzl\" (UID: \"b0f56f66-1cbf-40e8-bea8-edaa9473f4c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" Jun 10 15:06:13.677786 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.677718 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" Jun 10 15:06:13.824384 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:13.824207 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl"] Jun 10 15:06:13.826830 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:13.826788 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f56f66_1cbf_40e8_bea8_edaa9473f4c7.slice/crio-9df8fd5cf57ee8c3062e6c4be38468917f8087d267dbc06abe9a42b771d38328 WatchSource:0}: Error finding container 9df8fd5cf57ee8c3062e6c4be38468917f8087d267dbc06abe9a42b771d38328: Status 404 returned error can't find the container with id 9df8fd5cf57ee8c3062e6c4be38468917f8087d267dbc06abe9a42b771d38328 Jun 10 15:06:14.243272 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:14.243237 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" event={"ID":"b0f56f66-1cbf-40e8-bea8-edaa9473f4c7","Type":"ContainerStarted","Data":"9df8fd5cf57ee8c3062e6c4be38468917f8087d267dbc06abe9a42b771d38328"} Jun 10 15:06:15.248271 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.248188 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" event={"ID":"b0f56f66-1cbf-40e8-bea8-edaa9473f4c7","Type":"ContainerStarted","Data":"23668076a4fa661d6ca0ebf8797690d607c2b3c91ae78e89a5d0a1736dc44549"} Jun 10 15:06:15.248681 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.248450 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" Jun 10 15:06:15.253434 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.253412 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" Jun 10 15:06:15.272548 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.272499 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-tvwzl" podStartSLOduration=1.273670796 podStartE2EDuration="2.272483893s" totalImagesPullingTime="998.813097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:15.270703959 +0000 UTC m=+278.402996752" watchObservedRunningTime="2026-06-10 15:06:15.272483893 +0000 UTC m=+278.404776686" Jun 10 15:06:15.405333 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.405306 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:15.497128 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.497095 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:15.814060 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.814001 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-664b5f59d8-tdbdd"] Jun 10 15:06:15.818548 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.816857 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:15.820993 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.820967 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 10 15:06:15.821129 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.820966 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 10 15:06:15.821129 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.821017 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 10 15:06:15.821129 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.821066 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 10 15:06:15.821129 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.820972 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f5cnw\"" Jun 10 15:06:15.821129 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.820969 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 10 15:06:15.826878 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.826854 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664b5f59d8-tdbdd"] Jun 10 15:06:15.827620 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.827599 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:15.832769 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.832749 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 10 15:06:15.924023 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.923995 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:15.945220 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.945190 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-oauth-config\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:15.945365 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.945234 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g89m\" (UniqueName: \"kubernetes.io/projected/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-kube-api-access-7g89m\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:15.945365 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.945319 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-serving-cert\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:15.945365 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.945361 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-service-ca\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:15.945498 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.945429 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-oauth-serving-cert\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:15.945498 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.945456 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-config\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:15.945598 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:15.945562 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-trusted-ca-bundle\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.046074 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.046046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-serving-cert\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.046216 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.046084 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-service-ca\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.046276 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.046221 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-oauth-serving-cert\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.046276 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.046265 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-config\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.046382 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.046350 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-trusted-ca-bundle\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.046440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.046394 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-oauth-config\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.046492 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.046434 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g89m\" (UniqueName: \"kubernetes.io/projected/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-kube-api-access-7g89m\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.046800 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.046762 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-service-ca\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.047219 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.047195 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-config\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.047338 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.047306 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-oauth-serving-cert\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.047952 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.047916 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-trusted-ca-bundle\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.049115 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.049094 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-serving-cert\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.049375 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.049353 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-oauth-config\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.057346 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.057318 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g89m\" (UniqueName: \"kubernetes.io/projected/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-kube-api-access-7g89m\") pod \"console-664b5f59d8-tdbdd\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.130481 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.130424 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:16.278348 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:16.278274 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664b5f59d8-tdbdd"] Jun 10 15:06:16.281095 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:16.281050 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54cc10b3_d8d6_4ab0_982d_85c5e92a22db.slice/crio-41b3a7da4ed323878f3b13de331f5dc9ad24092dc2f5112108799576c1aedd19 WatchSource:0}: Error finding container 41b3a7da4ed323878f3b13de331f5dc9ad24092dc2f5112108799576c1aedd19: Status 404 returned error can't find the container with id 41b3a7da4ed323878f3b13de331f5dc9ad24092dc2f5112108799576c1aedd19 Jun 10 15:06:17.258773 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:17.258733 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664b5f59d8-tdbdd" event={"ID":"54cc10b3-d8d6-4ab0-982d-85c5e92a22db","Type":"ContainerStarted","Data":"41b3a7da4ed323878f3b13de331f5dc9ad24092dc2f5112108799576c1aedd19"} Jun 10 15:06:17.918805 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:17.918761 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" podUID="aff61f83-9ab8-403e-ad28-98afc53e8d00" Jun 10 15:06:17.918805 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:17.918761 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" podUID="75ab486a-c093-4fb3-bc99-0174f164ad3d" Jun 10 15:06:17.919278 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:17.918901 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nh957" podUID="ad836858-b9dc-4287-b3e8-907485054bf6" Jun 10 15:06:18.032919 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:18.032886 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:18.263723 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:18.263646 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:06:18.263879 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:18.263646 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nh957" Jun 10 15:06:18.263879 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:18.263661 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:06:20.272890 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.272859 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664b5f59d8-tdbdd" event={"ID":"54cc10b3-d8d6-4ab0-982d-85c5e92a22db","Type":"ContainerStarted","Data":"d5b3c562a51ac283cc8a46c671c7230b027c4c4809bb4192f900cd4fce62853f"} Jun 10 15:06:20.294630 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.294583 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-664b5f59d8-tdbdd" podStartSLOduration=2.219592324 podStartE2EDuration="5.294559234s" totalImagesPullingTime="3.07496691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:06:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:20.293233806 +0000 UTC m=+283.425526606" watchObservedRunningTime="2026-06-10 15:06:20.294559234 +0000 UTC m=+283.426852027" Jun 10 15:06:20.624732 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.624665 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:20.800130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.800085 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:20.976462 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.976388 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:20.995190 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.993918 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c"] Jun 10 15:06:20.997187 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.997114 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:20.998674 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.998653 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9jwp6"] Jun 10 15:06:20.999909 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.999880 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-c5csc\"" Jun 10 15:06:21.000037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.999939 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 10 15:06:21.000037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:20.999951 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 10 15:06:21.000223 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.000207 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 10 15:06:21.001529 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.001512 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.003708 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.003689 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 10 15:06:21.004394 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.004129 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 10 15:06:21.004394 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.004379 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 10 15:06:21.004620 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.004600 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-x5prf\"" Jun 10 15:06:21.011906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.011886 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c"] Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.088718 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-tls\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.088757 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffcck\" (UniqueName: \"kubernetes.io/projected/adf46b0f-5548-456d-8980-d56dd6ecb734-kube-api-access-ffcck\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.088811 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/adf46b0f-5548-456d-8980-d56dd6ecb734-root\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.088837 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/adf46b0f-5548-456d-8980-d56dd6ecb734-sys\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.088882 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn9xb\" (UniqueName: \"kubernetes.io/projected/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-kube-api-access-tn9xb\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.088914 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-textfile\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.088962 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adf46b0f-5548-456d-8980-d56dd6ecb734-metrics-client-ca\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.088997 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-accelerators-collector-config\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.089025 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.089051 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.089074 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.089101 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-wtmp\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.089279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.089126 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.106832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.106809 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:21.190231 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190207 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-tls\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.190347 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190243 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffcck\" (UniqueName: \"kubernetes.io/projected/adf46b0f-5548-456d-8980-d56dd6ecb734-kube-api-access-ffcck\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.190347 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190295 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/adf46b0f-5548-456d-8980-d56dd6ecb734-root\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.190347 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190319 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/adf46b0f-5548-456d-8980-d56dd6ecb734-sys\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.190557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190361 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn9xb\" (UniqueName: \"kubernetes.io/projected/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-kube-api-access-tn9xb\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.190557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190390 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-textfile\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.190557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190419 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adf46b0f-5548-456d-8980-d56dd6ecb734-metrics-client-ca\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.190557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190451 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-accelerators-collector-config\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.190557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190479 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.190557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190502 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.190557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190524 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.190557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190551 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-wtmp\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.191002 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.190583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.191002 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:21.190678 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 10 15:06:21.191002 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:21.190734 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-tls podName:adf46b0f-5548-456d-8980-d56dd6ecb734 nodeName:}" failed. No retries permitted until 2026-06-10 15:06:21.690716066 +0000 UTC m=+284.823008839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-tls") pod "node-exporter-9jwp6" (UID: "adf46b0f-5548-456d-8980-d56dd6ecb734") : secret "node-exporter-tls" not found Jun 10 15:06:21.191191 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.191005 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/adf46b0f-5548-456d-8980-d56dd6ecb734-sys\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.191191 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.191065 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/adf46b0f-5548-456d-8980-d56dd6ecb734-root\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.191191 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:21.191149 2561 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 10 15:06:21.191341 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:21.191198 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-tls podName:1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf nodeName:}" failed. No retries permitted until 2026-06-10 15:06:21.691181987 +0000 UTC m=+284.823474766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-bjq9c" (UID: "1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf") : secret "openshift-state-metrics-tls" not found Jun 10 15:06:21.191465 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.191444 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adf46b0f-5548-456d-8980-d56dd6ecb734-metrics-client-ca\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.192146 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.191670 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-accelerators-collector-config\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.192146 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.191891 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-textfile\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.192146 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.192042 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-wtmp\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.192356 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.192210 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.193370 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.193346 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.197296 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.197255 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.204270 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.204229 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffcck\" (UniqueName: \"kubernetes.io/projected/adf46b0f-5548-456d-8980-d56dd6ecb734-kube-api-access-ffcck\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.208372 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.208329 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn9xb\" (UniqueName: \"kubernetes.io/projected/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-kube-api-access-tn9xb\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.209443 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.209416 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:21.425480 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.425419 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:21.521882 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.521851 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:21.696563 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.696460 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:06:21.696563 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.696526 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:21.696772 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.696572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:06:21.696772 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.696603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:06:21.696772 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.696627 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-tls\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:21.696772 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:21.696748 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 10 15:06:21.696995 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:21.696805 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-tls podName:adf46b0f-5548-456d-8980-d56dd6ecb734 nodeName:}" failed. No retries permitted until 2026-06-10 15:06:22.696786847 +0000 UTC m=+285.829079620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-tls") pod "node-exporter-9jwp6" (UID: "adf46b0f-5548-456d-8980-d56dd6ecb734") : secret "node-exporter-tls" not found Jun 10 15:06:21.697355 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:21.697265 2561 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 10 15:06:21.697355 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:21.697324 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-tls podName:1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf nodeName:}" failed. No retries permitted until 2026-06-10 15:06:22.697307609 +0000 UTC m=+285.829600387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-bjq9c" (UID: "1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf") : secret "openshift-state-metrics-tls" not found Jun 10 15:06:21.700094 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.699994 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad836858-b9dc-4287-b3e8-907485054bf6-metrics-tls\") pod \"dns-default-nh957\" (UID: \"ad836858-b9dc-4287-b3e8-907485054bf6\") " pod="openshift-dns/dns-default-nh957" Jun 10 15:06:21.700904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.700877 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75ab486a-c093-4fb3-bc99-0174f164ad3d-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-c8w7n\" (UID: \"75ab486a-c093-4fb3-bc99-0174f164ad3d\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:06:21.701198 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.701156 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"image-registry-ddf7f6777-mnb8x\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:06:21.771971 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.771922 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:21.867220 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.867197 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nnbbk\"" Jun 10 15:06:21.867220 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.867197 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-982f9\"" Jun 10 15:06:21.867422 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.867197 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pscgt\"" Jun 10 15:06:21.875209 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.875190 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nh957" Jun 10 15:06:21.875331 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.875273 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" Jun 10 15:06:21.875392 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.875332 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:06:21.876488 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:21.876462 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:22.061634 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:22.061595 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:22.214193 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:22.214156 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:22.360852 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:22.360772 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:22.704672 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:22.704581 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:22.704672 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:22.704638 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-tls\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:22.707475 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:22.707453 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/adf46b0f-5548-456d-8980-d56dd6ecb734-node-exporter-tls\") pod \"node-exporter-9jwp6\" (UID: \"adf46b0f-5548-456d-8980-d56dd6ecb734\") " pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:22.707679 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:22.707656 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-bjq9c\" (UID: \"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:22.807975 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:22.807942 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" Jun 10 15:06:22.812556 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:22.812533 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9jwp6" Jun 10 15:06:23.875638 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:23.875603 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:24.006078 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.006033 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:24.062144 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.062113 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c97668995-dwfmm"] Jun 10 15:06:24.065327 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.065305 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.068171 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.068147 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2ve9r5b3hslff\"" Jun 10 15:06:24.068626 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.068600 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 10 15:06:24.073170 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.073146 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 10 15:06:24.073293 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.073226 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 10 15:06:24.073293 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.073280 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-b89hg\"" Jun 10 15:06:24.073407 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.073153 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 10 15:06:24.073517 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.073501 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 10 15:06:24.101406 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.101371 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c97668995-dwfmm"] Jun 10 15:06:24.205706 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.205635 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:24.219894 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.219864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.220029 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.219914 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nhsv\" (UniqueName: \"kubernetes.io/projected/3a00bda7-20f0-4410-bed3-05190508710d-kube-api-access-2nhsv\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.220029 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.219964 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.220158 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.220050 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.220158 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.220083 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-grpc-tls\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.220261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.220171 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-tls\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.220261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.220199 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a00bda7-20f0-4410-bed3-05190508710d-metrics-client-ca\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.220363 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.220254 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.320897 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.320866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-tls\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.321061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.320903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a00bda7-20f0-4410-bed3-05190508710d-metrics-client-ca\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.321061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.320953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.321061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.321008 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.321061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.321046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nhsv\" (UniqueName: \"kubernetes.io/projected/3a00bda7-20f0-4410-bed3-05190508710d-kube-api-access-2nhsv\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.321260 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.321073 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.321260 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.321109 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.321260 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.321132 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-grpc-tls\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.321880 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.321711 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a00bda7-20f0-4410-bed3-05190508710d-metrics-client-ca\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.324050 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.324005 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.324143 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.324049 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-tls\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.324712 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.324695 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.324822 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.324793 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.325164 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.325146 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-grpc-tls\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.325406 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.325382 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3a00bda7-20f0-4410-bed3-05190508710d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.332325 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.332307 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nhsv\" (UniqueName: \"kubernetes.io/projected/3a00bda7-20f0-4410-bed3-05190508710d-kube-api-access-2nhsv\") pod \"thanos-querier-7c97668995-dwfmm\" (UID: \"3a00bda7-20f0-4410-bed3-05190508710d\") " pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.379381 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.379355 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:24.818309 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:24.818261 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:25.495666 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.495632 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:25.676083 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.676030 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:25.694356 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.694323 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-szx86"] Jun 10 15:06:25.697264 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.697243 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:25.700037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.700009 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8cjfz\"" Jun 10 15:06:25.700140 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.700009 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 10 15:06:25.710044 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.710024 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-szx86"] Jun 10 15:06:25.731397 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.731373 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64cdd78d57-45tgx"] Jun 10 15:06:25.733744 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.733724 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.748381 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.748319 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cdd78d57-45tgx"] Jun 10 15:06:25.787979 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.787954 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:25.835002 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.834973 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c62d43e1-bd5b-48a0-951e-73693f36dedf-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-szx86\" (UID: \"c62d43e1-bd5b-48a0-951e-73693f36dedf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:25.835110 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.835011 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twqqw\" (UniqueName: \"kubernetes.io/projected/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-kube-api-access-twqqw\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.835110 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.835058 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-oauth-serving-cert\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.835110 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.835087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-oauth-config\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.835266 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.835117 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zq76\" (UniqueName: \"kubernetes.io/projected/c62d43e1-bd5b-48a0-951e-73693f36dedf-kube-api-access-2zq76\") pod \"monitoring-plugin-76b68cc874-szx86\" (UID: \"c62d43e1-bd5b-48a0-951e-73693f36dedf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:25.835266 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.835228 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-service-ca\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.835368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.835272 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-serving-cert\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.835368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.835311 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-config\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.835368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.835359 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-trusted-ca-bundle\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.936461 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.936429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-service-ca\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.936592 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.936477 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-serving-cert\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.936592 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.936498 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-config\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.936592 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.936516 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-trusted-ca-bundle\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.936592 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.936579 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c62d43e1-bd5b-48a0-951e-73693f36dedf-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-szx86\" (UID: \"c62d43e1-bd5b-48a0-951e-73693f36dedf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:25.936800 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.936611 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twqqw\" (UniqueName: \"kubernetes.io/projected/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-kube-api-access-twqqw\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.936800 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.936637 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-oauth-serving-cert\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.936800 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.936659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-oauth-config\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.936800 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.936684 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zq76\" (UniqueName: \"kubernetes.io/projected/c62d43e1-bd5b-48a0-951e-73693f36dedf-kube-api-access-2zq76\") pod \"monitoring-plugin-76b68cc874-szx86\" (UID: \"c62d43e1-bd5b-48a0-951e-73693f36dedf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:25.937382 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:25.937075 2561 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 10 15:06:25.937382 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:25.937147 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c62d43e1-bd5b-48a0-951e-73693f36dedf-monitoring-plugin-cert podName:c62d43e1-bd5b-48a0-951e-73693f36dedf nodeName:}" failed. No retries permitted until 2026-06-10 15:06:26.437130158 +0000 UTC m=+289.569422930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/c62d43e1-bd5b-48a0-951e-73693f36dedf-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-szx86" (UID: "c62d43e1-bd5b-48a0-951e-73693f36dedf") : secret "monitoring-plugin-cert" not found Jun 10 15:06:25.937916 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.937854 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-config\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.937916 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.937856 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-oauth-serving-cert\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.938116 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.937954 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-trusted-ca-bundle\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.938116 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.938010 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-service-ca\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.940018 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.939977 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-serving-cert\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.940175 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.940132 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-oauth-config\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.956778 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.956737 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twqqw\" (UniqueName: \"kubernetes.io/projected/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-kube-api-access-twqqw\") pod \"console-64cdd78d57-45tgx\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:25.957453 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:25.957421 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zq76\" (UniqueName: \"kubernetes.io/projected/c62d43e1-bd5b-48a0-951e-73693f36dedf-kube-api-access-2zq76\") pod \"monitoring-plugin-76b68cc874-szx86\" (UID: \"c62d43e1-bd5b-48a0-951e-73693f36dedf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:26.045004 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:26.044984 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:26.130732 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:26.130703 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:26.130864 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:26.130766 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:26.132317 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:26.132280 2561 patch_prober.go:28] interesting pod/console-664b5f59d8-tdbdd container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused" start-of-body= Jun 10 15:06:26.132430 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:26.132344 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-664b5f59d8-tdbdd" podUID="54cc10b3-d8d6-4ab0-982d-85c5e92a22db" containerName="console" probeResult="failure" output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused" Jun 10 15:06:26.170813 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:26.170761 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:26.441508 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:26.441413 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c62d43e1-bd5b-48a0-951e-73693f36dedf-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-szx86\" (UID: \"c62d43e1-bd5b-48a0-951e-73693f36dedf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:26.443945 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:26.443910 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c62d43e1-bd5b-48a0-951e-73693f36dedf-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-szx86\" (UID: \"c62d43e1-bd5b-48a0-951e-73693f36dedf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:26.608944 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:26.608894 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:27.240585 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:27.240549 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:28.637141 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:28.637106 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadf46b0f_5548_456d_8980_d56dd6ecb734.slice/crio-28bbdf1496774da32d52110aff4fe189bf83aa81e79d17c329b43bb2c98e8d8f WatchSource:0}: Error finding container 28bbdf1496774da32d52110aff4fe189bf83aa81e79d17c329b43bb2c98e8d8f: Status 404 returned error can't find the container with id 28bbdf1496774da32d52110aff4fe189bf83aa81e79d17c329b43bb2c98e8d8f Jun 10 15:06:28.855491 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:28.855319 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-szx86"] Jun 10 15:06:28.858343 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:28.858290 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc62d43e1_bd5b_48a0_951e_73693f36dedf.slice/crio-3b2c3f7f5c560de1e99d27e492044f13cbd5ff5b368e87a121f92020c8cab0cc WatchSource:0}: Error finding container 3b2c3f7f5c560de1e99d27e492044f13cbd5ff5b368e87a121f92020c8cab0cc: Status 404 returned error can't find the container with id 3b2c3f7f5c560de1e99d27e492044f13cbd5ff5b368e87a121f92020c8cab0cc Jun 10 15:06:29.056827 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.056796 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n"] Jun 10 15:06:29.059782 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:29.059740 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ab486a_c093_4fb3_bc99_0174f164ad3d.slice/crio-ceeff45a56e7ee2ada31f9bff1966d02f59e2ea81daf05dc30660e16ec7a2cfb WatchSource:0}: Error finding container ceeff45a56e7ee2ada31f9bff1966d02f59e2ea81daf05dc30660e16ec7a2cfb: Status 404 returned error can't find the container with id ceeff45a56e7ee2ada31f9bff1966d02f59e2ea81daf05dc30660e16ec7a2cfb Jun 10 15:06:29.059901 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.059819 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c"] Jun 10 15:06:29.063448 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:29.063423 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ccfa0d7_d2cb_40e5_8cbd_b1075ade2fdf.slice/crio-0addea81257ede326e6087078333eb564c09659815e8a5845d41877dd9867bfe WatchSource:0}: Error finding container 0addea81257ede326e6087078333eb564c09659815e8a5845d41877dd9867bfe: Status 404 returned error can't find the container with id 0addea81257ede326e6087078333eb564c09659815e8a5845d41877dd9867bfe Jun 10 15:06:29.113401 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.112591 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cdd78d57-45tgx"] Jun 10 15:06:29.114232 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.114193 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nh957"] Jun 10 15:06:29.116434 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.116418 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ddf7f6777-mnb8x"] Jun 10 15:06:29.116676 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:29.116602 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad836858_b9dc_4287_b3e8_907485054bf6.slice/crio-83c23720d97ea380af3b949a9d582069c090279e4091d88845ee782b17f13311 WatchSource:0}: Error finding container 83c23720d97ea380af3b949a9d582069c090279e4091d88845ee782b17f13311: Status 404 returned error can't find the container with id 83c23720d97ea380af3b949a9d582069c090279e4091d88845ee782b17f13311 Jun 10 15:06:29.117974 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.117950 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c97668995-dwfmm"] Jun 10 15:06:29.118794 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:06:29.118697 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6a9c01_e824_4c4e_8e83_8dc501fc2c79.slice/crio-37497a8765c72ecdabdf11eef8cc392e84df80b9df4fe8fbbc84ee5af1fcd181 WatchSource:0}: Error finding container 37497a8765c72ecdabdf11eef8cc392e84df80b9df4fe8fbbc84ee5af1fcd181: Status 404 returned error can't find the container with id 37497a8765c72ecdabdf11eef8cc392e84df80b9df4fe8fbbc84ee5af1fcd181 Jun 10 15:06:29.315971 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.315849 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" event={"ID":"3a00bda7-20f0-4410-bed3-05190508710d","Type":"ContainerStarted","Data":"b7c2bb95606fb67f2a5386ebc70b402f0efc154bfbea10027c9cc2cbfc83c79e"} Jun 10 15:06:29.323027 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.322995 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" event={"ID":"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf","Type":"ContainerStarted","Data":"efbd6b5e40da5dcf0e1412b08de6b111b646e4de1ca32950b6d3ed24c53245b2"} Jun 10 15:06:29.323468 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.323443 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" event={"ID":"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf","Type":"ContainerStarted","Data":"347992331afe959c9d8266827f568a56b1bcc3a5c7413ad575ba78c847d0978d"} Jun 10 15:06:29.323839 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.323820 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" event={"ID":"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf","Type":"ContainerStarted","Data":"0addea81257ede326e6087078333eb564c09659815e8a5845d41877dd9867bfe"} Jun 10 15:06:29.325021 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.324999 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cdd78d57-45tgx" event={"ID":"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79","Type":"ContainerStarted","Data":"9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e"} Jun 10 15:06:29.325486 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.325466 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cdd78d57-45tgx" event={"ID":"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79","Type":"ContainerStarted","Data":"37497a8765c72ecdabdf11eef8cc392e84df80b9df4fe8fbbc84ee5af1fcd181"} Jun 10 15:06:29.327285 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.327245 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" event={"ID":"c62d43e1-bd5b-48a0-951e-73693f36dedf","Type":"ContainerStarted","Data":"3b2c3f7f5c560de1e99d27e492044f13cbd5ff5b368e87a121f92020c8cab0cc"} Jun 10 15:06:29.329225 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.329200 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-bjzrv" event={"ID":"0f451555-80a8-4b4b-a876-9b49611345f6","Type":"ContainerStarted","Data":"131c982dba05fdfcbd7f14c1f5fd5a7e065dfe78cdde69c12923ae50caf87ded"} Jun 10 15:06:29.329986 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.329967 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-bjzrv" Jun 10 15:06:29.334050 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.334028 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9jwp6" event={"ID":"adf46b0f-5548-456d-8980-d56dd6ecb734","Type":"ContainerStarted","Data":"28bbdf1496774da32d52110aff4fe189bf83aa81e79d17c329b43bb2c98e8d8f"} Jun 10 15:06:29.334618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.334602 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-bjzrv" Jun 10 15:06:29.336853 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.336834 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" event={"ID":"75ab486a-c093-4fb3-bc99-0174f164ad3d","Type":"ContainerStarted","Data":"ceeff45a56e7ee2ada31f9bff1966d02f59e2ea81daf05dc30660e16ec7a2cfb"} Jun 10 15:06:29.347284 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.347265 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:06:29.347390 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.347305 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" event={"ID":"aff61f83-9ab8-403e-ad28-98afc53e8d00","Type":"ContainerStarted","Data":"997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855"} Jun 10 15:06:29.347390 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.347329 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" event={"ID":"aff61f83-9ab8-403e-ad28-98afc53e8d00","Type":"ContainerStarted","Data":"827361b2779159e137cc5ec56462a36ed09d02cbc780f67929ec68c99971fdf9"} Jun 10 15:06:29.347390 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.347349 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nh957" event={"ID":"ad836858-b9dc-4287-b3e8-907485054bf6","Type":"ContainerStarted","Data":"83c23720d97ea380af3b949a9d582069c090279e4091d88845ee782b17f13311"} Jun 10 15:06:29.356326 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.356273 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64cdd78d57-45tgx" podStartSLOduration=4.356261004 podStartE2EDuration="4.356261004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:29.355707813 +0000 UTC m=+292.488000610" watchObservedRunningTime="2026-06-10 15:06:29.356261004 +0000 UTC m=+292.488553795" Jun 10 15:06:29.386094 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.385493 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" podStartSLOduration=270.385474218 podStartE2EDuration="4m30.385474218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:29.383622539 +0000 UTC m=+292.515915329" watchObservedRunningTime="2026-06-10 15:06:29.385474218 +0000 UTC m=+292.517767013" Jun 10 15:06:29.408465 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:29.408092 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-bjzrv" podStartSLOduration=1.8380460090000001 podStartE2EDuration="20.408073903s" totalImagesPullingTime="18.570027894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:06:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:29.405164945 +0000 UTC m=+292.537457741" watchObservedRunningTime="2026-06-10 15:06:29.408073903 +0000 UTC m=+292.540366679" Jun 10 15:06:30.290329 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:30.289778 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664b5f59d8-tdbdd"] Jun 10 15:06:33.339050 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:33.339000 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-msxf9" podUID="6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce" Jun 10 15:06:33.966128 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:33.966009 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" podUID="163ce46a-ece4-42b3-8d69-ba8a9d03ff22" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.10:8000/readyz\": dial tcp 10.132.0.10:8000: connect: connection refused" Jun 10 15:06:34.368351 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.367867 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" event={"ID":"c62d43e1-bd5b-48a0-951e-73693f36dedf","Type":"ContainerStarted","Data":"25650eac3248e1ecda8352d8e56bd2c9c75cd3eedb3ae2fc2b3c79390b1f7e90"} Jun 10 15:06:34.368351 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.368099 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:34.369805 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.369772 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9jwp6" event={"ID":"adf46b0f-5548-456d-8980-d56dd6ecb734","Type":"ContainerStarted","Data":"dccfde817e8ecc6a4bcbb4d958343cfa1ba6172b28289a6161eb0edd1fded9e0"} Jun 10 15:06:34.372186 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.372143 2561 generic.go:356] "Generic (PLEG): container finished" podID="163ce46a-ece4-42b3-8d69-ba8a9d03ff22" containerID="d5b623939a6df115b5a752467a2f02c579f883674c4327d44a1acbae88552efd" exitCode=1 Jun 10 15:06:34.372295 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.372244 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" event={"ID":"163ce46a-ece4-42b3-8d69-ba8a9d03ff22","Type":"ContainerDied","Data":"d5b623939a6df115b5a752467a2f02c579f883674c4327d44a1acbae88552efd"} Jun 10 15:06:34.372295 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.372291 2561 scope.go:117] "RemoveContainer" containerID="b37bed4d715bdaafac2fdc66843be006636b28573910ddbb44d31fbad555f322" Jun 10 15:06:34.372685 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.372659 2561 scope.go:117] "RemoveContainer" containerID="d5b623939a6df115b5a752467a2f02c579f883674c4327d44a1acbae88552efd" Jun 10 15:06:34.372887 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:34.372851 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-6bbf87b647-s2b72_open-cluster-management-agent-addon(163ce46a-ece4-42b3-8d69-ba8a9d03ff22)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" podUID="163ce46a-ece4-42b3-8d69-ba8a9d03ff22" Jun 10 15:06:34.374095 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.374069 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" Jun 10 15:06:34.374579 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.374553 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" event={"ID":"3a00bda7-20f0-4410-bed3-05190508710d","Type":"ContainerStarted","Data":"12f87aed6bc6cf13f07bf76fbd7e4708a965164b6895201ed6982623493df78d"} Jun 10 15:06:34.420284 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:34.420052 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-szx86" podStartSLOduration=4.149024515 podStartE2EDuration="9.420034238s" totalImagesPullingTime="5.271009723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:34.38861832 +0000 UTC m=+297.520911116" watchObservedRunningTime="2026-06-10 15:06:34.420034238 +0000 UTC m=+297.552327034" Jun 10 15:06:35.382290 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.382251 2561 generic.go:356] "Generic (PLEG): container finished" podID="adf46b0f-5548-456d-8980-d56dd6ecb734" containerID="dccfde817e8ecc6a4bcbb4d958343cfa1ba6172b28289a6161eb0edd1fded9e0" exitCode=0 Jun 10 15:06:35.382755 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.382527 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9jwp6" event={"ID":"adf46b0f-5548-456d-8980-d56dd6ecb734","Type":"ContainerDied","Data":"dccfde817e8ecc6a4bcbb4d958343cfa1ba6172b28289a6161eb0edd1fded9e0"} Jun 10 15:06:35.387277 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.387251 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" event={"ID":"75ab486a-c093-4fb3-bc99-0174f164ad3d","Type":"ContainerStarted","Data":"034e511b73007068f9dae28f13a9e89e6cd6f1d7480c5b3597ccc95e952ace03"} Jun 10 15:06:35.391134 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.391108 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nh957" event={"ID":"ad836858-b9dc-4287-b3e8-907485054bf6","Type":"ContainerStarted","Data":"6d82e1a3e0953e82d52496af78d11050f88a313ed9612d2ad921e5c0f99f2c7b"} Jun 10 15:06:35.391241 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.391157 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nh957" event={"ID":"ad836858-b9dc-4287-b3e8-907485054bf6","Type":"ContainerStarted","Data":"20286bc5d66c96d4e0c264f845990452a0dd3faa91f0a88ccbc58e293bd21952"} Jun 10 15:06:35.394270 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.394224 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" event={"ID":"3a00bda7-20f0-4410-bed3-05190508710d","Type":"ContainerStarted","Data":"6ad49574b40435506d04c110b9159407300d064e08f1d3ef2a85973faa7775f6"} Jun 10 15:06:35.394394 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.394288 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" event={"ID":"3a00bda7-20f0-4410-bed3-05190508710d","Type":"ContainerStarted","Data":"fe848ad0fe536c1d18cbcea87f264eb99478838c7b6eacbb2b7b34bb13d94d1b"} Jun 10 15:06:35.397145 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.397119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" event={"ID":"1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf","Type":"ContainerStarted","Data":"af9c4f0304bdeeb3397048985a17d2115d931757811b63bb12fe3e14346ddfc3"} Jun 10 15:06:35.430355 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.430315 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nh957" podStartSLOduration=259.412333242 podStartE2EDuration="4m24.430298348s" totalImagesPullingTime="5.017965106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:35.429092778 +0000 UTC m=+298.561385573" watchObservedRunningTime="2026-06-10 15:06:35.430298348 +0000 UTC m=+298.562591143" Jun 10 15:06:35.448052 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.448011 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-c8w7n" podStartSLOduration=272.378365425 podStartE2EDuration="4m37.447997522s" totalImagesPullingTime="5.069632097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:35.447374082 +0000 UTC m=+298.579666878" watchObservedRunningTime="2026-06-10 15:06:35.447997522 +0000 UTC m=+298.580290366" Jun 10 15:06:35.475851 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.475811 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-bjq9c" podStartSLOduration=10.305882965 podStartE2EDuration="15.475792891s" totalImagesPullingTime="5.169909926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:35.475640491 +0000 UTC m=+298.607933298" watchObservedRunningTime="2026-06-10 15:06:35.475792891 +0000 UTC m=+298.608085686" Jun 10 15:06:35.580511 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.580473 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:35.662415 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:35.662342 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:36.046196 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.046160 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:36.046367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.046231 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:36.047758 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.047731 2561 patch_prober.go:28] interesting pod/console-64cdd78d57-45tgx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.23:8443/health\": dial tcp 10.132.0.23:8443: connect: connection refused" start-of-body= Jun 10 15:06:36.047895 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.047774 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-64cdd78d57-45tgx" podUID="ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" containerName="console" probeResult="failure" output="Get \"https://10.132.0.23:8443/health\": dial tcp 10.132.0.23:8443: connect: connection refused" Jun 10 15:06:36.405502 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.405467 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" event={"ID":"3a00bda7-20f0-4410-bed3-05190508710d","Type":"ContainerStarted","Data":"383ef1c9b0a8b51143331bed6b9fbe7d9e14cd7d76a9a40fca885efa899f21b7"} Jun 10 15:06:36.405884 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.405582 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" event={"ID":"3a00bda7-20f0-4410-bed3-05190508710d","Type":"ContainerStarted","Data":"155c7ba6a7d755c60bda3f12881adb2f05903397ba7f578fd98ad5cb284f5072"} Jun 10 15:06:36.408090 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.408061 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9jwp6" event={"ID":"adf46b0f-5548-456d-8980-d56dd6ecb734","Type":"ContainerStarted","Data":"c6c08462fb60bce0e6920e5ed6fa19016279450931a17053d571b9a633593ea8"} Jun 10 15:06:36.408187 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.408119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9jwp6" event={"ID":"adf46b0f-5548-456d-8980-d56dd6ecb734","Type":"ContainerStarted","Data":"989bf1645b7438f201b6ea414f496d491080ff8a91dfbb66ba3bdc705603949e"} Jun 10 15:06:36.409115 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.409093 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nh957" Jun 10 15:06:36.442076 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.442022 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9jwp6" podStartSLOduration=10.871956196 podStartE2EDuration="16.442008769s" totalImagesPullingTime="5.462472395s" totalInitContainerRuntime="107.580178ms" isStatefulPod=true podCreationTimestamp="2026-06-10 15:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:36.44056974 +0000 UTC m=+299.572862546" watchObservedRunningTime="2026-06-10 15:06:36.442008769 +0000 UTC m=+299.574301563" Jun 10 15:06:36.500513 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.500487 2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:06:36.500869 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:36.500849 2561 scope.go:117] "RemoveContainer" containerID="d5b623939a6df115b5a752467a2f02c579f883674c4327d44a1acbae88552efd" Jun 10 15:06:36.501094 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:06:36.501066 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-6bbf87b647-s2b72_open-cluster-management-agent-addon(163ce46a-ece4-42b3-8d69-ba8a9d03ff22)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" podUID="163ce46a-ece4-42b3-8d69-ba8a9d03ff22" Jun 10 15:06:37.313906 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:37.313882 2561 kubelet.go:1648] "Image garbage collection succeeded" Jun 10 15:06:37.421577 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:37.421387 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" event={"ID":"3a00bda7-20f0-4410-bed3-05190508710d","Type":"ContainerStarted","Data":"395e3602a3cb585ce29ce1e4c4ddf4dc0d9b7fc36e978e8d7197e8b9176c1946"} Jun 10 15:06:37.574373 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:37.421721 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:37.574373 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:37.459142 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" podStartSLOduration=7.075997298 podStartE2EDuration="13.459125265s" totalImagesPullingTime="6.383127967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:06:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:06:37.456283763 +0000 UTC m=+300.588576579" watchObservedRunningTime="2026-06-10 15:06:37.459125265 +0000 UTC m=+300.591418060" Jun 10 15:06:38.038387 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:38.038317 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:41.234902 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:41.234866 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:42.128397 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:42.128367 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ddf7f6777-mnb8x"] Jun 10 15:06:42.469701 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:42.469589 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:42.541215 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:42.541192 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:43.432429 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:43.432401 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c97668995-dwfmm" Jun 10 15:06:43.965514 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:43.965482 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:06:43.965889 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:43.965811 2561 scope.go:117] "RemoveContainer" containerID="d5b623939a6df115b5a752467a2f02c579f883674c4327d44a1acbae88552efd" Jun 10 15:06:44.337670 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:44.337642 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:06:44.450458 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:44.450433 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" event={"ID":"163ce46a-ece4-42b3-8d69-ba8a9d03ff22","Type":"ContainerStarted","Data":"3442a12b60c766acb7d881cbc7ce2092dec008d339f2d23f2e369dca93a6c87f"} Jun 10 15:06:44.450708 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:44.450692 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:06:44.452235 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:44.452213 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bbf87b647-s2b72" Jun 10 15:06:45.306661 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:45.306625 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:45.403752 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:45.403711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:46.049489 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:46.049465 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:46.053166 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:46.053144 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:06:46.342127 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:46.342056 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:46.423493 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:46.423472 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nh957" Jun 10 15:06:46.476065 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:46.476032 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:47.623567 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:47.623535 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:48.656693 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:48.656661 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:48.764195 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:48.764161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:48.798634 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:48.798609 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-d67p5_516ba81a-80cd-49fe-bf46-f1eb09dae80b/cluster-monitoring-operator/0.log" Jun 10 15:06:49.794572 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:49.794542 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-szx86_c62d43e1-bd5b-48a0-951e-73693f36dedf/monitoring-plugin/0.log" Jun 10 15:06:49.999145 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:49.999122 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9jwp6_adf46b0f-5548-456d-8980-d56dd6ecb734/init-textfile/0.log" Jun 10 15:06:50.192054 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:50.191981 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9jwp6_adf46b0f-5548-456d-8980-d56dd6ecb734/node-exporter/0.log" Jun 10 15:06:50.390880 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:50.390858 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9jwp6_adf46b0f-5548-456d-8980-d56dd6ecb734/kube-rbac-proxy/0.log" Jun 10 15:06:50.754778 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:50.754750 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64cdd78d57-45tgx"] Jun 10 15:06:50.933963 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:50.933917 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:51.045965 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:51.045917 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:51.171679 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:51.171648 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:51.791360 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:51.791340 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-bjq9c_1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf/kube-rbac-proxy-main/0.log" Jun 10 15:06:51.991135 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:51.991114 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-bjq9c_1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf/kube-rbac-proxy-self/0.log" Jun 10 15:06:52.132646 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:52.132597 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:06:52.192055 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:52.192032 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-bjq9c_1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf/openshift-state-metrics/0.log" Jun 10 15:06:54.195599 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:54.195570 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-tvwzl_b0f56f66-1cbf-40e8-bea8-edaa9473f4c7/prometheus-operator-admission-webhook/0.log" Jun 10 15:06:54.992434 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:54.992407 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/thanos-query/0.log" Jun 10 15:06:55.193641 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.193601 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/kube-rbac-proxy-web/0.log" Jun 10 15:06:55.320530 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.320488 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-664b5f59d8-tdbdd" podUID="54cc10b3-d8d6-4ab0-982d-85c5e92a22db" containerName="console" containerID="cri-o://d5b3c562a51ac283cc8a46c671c7230b027c4c4809bb4192f900cd4fce62853f" gracePeriod=15 Jun 10 15:06:55.393325 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.393305 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/kube-rbac-proxy/0.log" Jun 10 15:06:55.492868 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.492844 2561 generic.go:356] "Generic (PLEG): container finished" podID="54cc10b3-d8d6-4ab0-982d-85c5e92a22db" containerID="d5b3c562a51ac283cc8a46c671c7230b027c4c4809bb4192f900cd4fce62853f" exitCode=0 Jun 10 15:06:55.493000 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.492919 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664b5f59d8-tdbdd" event={"ID":"54cc10b3-d8d6-4ab0-982d-85c5e92a22db","Type":"ContainerDied","Data":"d5b3c562a51ac283cc8a46c671c7230b027c4c4809bb4192f900cd4fce62853f"} Jun 10 15:06:55.560704 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.560683 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:55.587816 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.587755 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-serving-cert\") pod \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " Jun 10 15:06:55.587816 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.587796 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-config\") pod \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " Jun 10 15:06:55.588017 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.587850 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g89m\" (UniqueName: \"kubernetes.io/projected/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-kube-api-access-7g89m\") pod \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " Jun 10 15:06:55.588017 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.587874 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-trusted-ca-bundle\") pod \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " Jun 10 15:06:55.588017 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.587962 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-oauth-serving-cert\") pod \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " Jun 10 15:06:55.588017 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.587994 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-service-ca\") pod \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " Jun 10 15:06:55.588206 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.588043 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-oauth-config\") pod \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\" (UID: \"54cc10b3-d8d6-4ab0-982d-85c5e92a22db\") " Jun 10 15:06:55.588495 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.588452 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-config" (OuterVolumeSpecName: "console-config") pod "54cc10b3-d8d6-4ab0-982d-85c5e92a22db" (UID: "54cc10b3-d8d6-4ab0-982d-85c5e92a22db"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:06:55.588697 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.588507 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "54cc10b3-d8d6-4ab0-982d-85c5e92a22db" (UID: "54cc10b3-d8d6-4ab0-982d-85c5e92a22db"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:06:55.588697 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.588569 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "54cc10b3-d8d6-4ab0-982d-85c5e92a22db" (UID: "54cc10b3-d8d6-4ab0-982d-85c5e92a22db"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:06:55.589056 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.589033 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-service-ca" (OuterVolumeSpecName: "service-ca") pod "54cc10b3-d8d6-4ab0-982d-85c5e92a22db" (UID: "54cc10b3-d8d6-4ab0-982d-85c5e92a22db"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:06:55.590890 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.590860 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "54cc10b3-d8d6-4ab0-982d-85c5e92a22db" (UID: "54cc10b3-d8d6-4ab0-982d-85c5e92a22db"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 15:06:55.591021 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.590897 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-kube-api-access-7g89m" (OuterVolumeSpecName: "kube-api-access-7g89m") pod "54cc10b3-d8d6-4ab0-982d-85c5e92a22db" (UID: "54cc10b3-d8d6-4ab0-982d-85c5e92a22db"). InnerVolumeSpecName "kube-api-access-7g89m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:06:55.591021 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.591003 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "54cc10b3-d8d6-4ab0-982d-85c5e92a22db" (UID: "54cc10b3-d8d6-4ab0-982d-85c5e92a22db"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 15:06:55.592867 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.592848 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/prom-label-proxy/0.log" Jun 10 15:06:55.647084 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.647059 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:55.689248 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.689230 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g89m\" (UniqueName: \"kubernetes.io/projected/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-kube-api-access-7g89m\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:06:55.689248 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.689247 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-trusted-ca-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:06:55.689351 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.689256 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-oauth-serving-cert\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:06:55.689351 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.689265 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-service-ca\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:06:55.689351 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.689273 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-oauth-config\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:06:55.689351 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.689281 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-serving-cert\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:06:55.689351 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.689289 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54cc10b3-d8d6-4ab0-982d-85c5e92a22db-console-config\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:06:55.791055 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.791036 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/kube-rbac-proxy-rules/0.log" Jun 10 15:06:55.991688 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:55.991633 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/kube-rbac-proxy-metrics/0.log" Jun 10 15:06:56.191112 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:56.191088 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-c8w7n_75ab486a-c093-4fb3-bc99-0174f164ad3d/networking-console-plugin/0.log" Jun 10 15:06:56.498616 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:56.498596 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664b5f59d8-tdbdd" Jun 10 15:06:56.498616 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:56.498604 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664b5f59d8-tdbdd" event={"ID":"54cc10b3-d8d6-4ab0-982d-85c5e92a22db","Type":"ContainerDied","Data":"41b3a7da4ed323878f3b13de331f5dc9ad24092dc2f5112108799576c1aedd19"} Jun 10 15:06:56.499010 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:56.498650 2561 scope.go:117] "RemoveContainer" containerID="d5b3c562a51ac283cc8a46c671c7230b027c4c4809bb4192f900cd4fce62853f" Jun 10 15:06:56.523655 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:56.523633 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664b5f59d8-tdbdd"] Jun 10 15:06:56.528604 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:56.528584 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-664b5f59d8-tdbdd"] Jun 10 15:06:56.991960 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:56.991924 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64cdd78d57-45tgx_ac6a9c01-e824-4c4e-8e83-8dc501fc2c79/console/0.log" Jun 10 15:06:57.346270 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:57.346243 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54cc10b3-d8d6-4ab0-982d-85c5e92a22db" path="/var/lib/kubelet/pods/54cc10b3-d8d6-4ab0-982d-85c5e92a22db/volumes" Jun 10 15:06:57.592294 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:57.592273 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-bjzrv_0f451555-80a8-4b4b-a876-9b49611345f6/download-server/0.log" Jun 10 15:06:58.596218 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:58.596196 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nh957_ad836858-b9dc-4287-b3e8-907485054bf6/dns/0.log" Jun 10 15:06:58.793686 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:58.793668 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nh957_ad836858-b9dc-4287-b3e8-907485054bf6/kube-rbac-proxy/0.log" Jun 10 15:06:58.926462 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:58.926390 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:59.023296 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:59.023276 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:06:59.192535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:06:59.192481 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-djm7n_e512dd9b-89b7-4bb0-abef-1d717f999ddc/dns-node-resolver/0.log" Jun 10 15:07:07.147415 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.147357 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" podUID="aff61f83-9ab8-403e-ad28-98afc53e8d00" containerName="registry" containerID="cri-o://997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855" gracePeriod=30 Jun 10 15:07:07.415056 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.415003 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:07:07.487778 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.487755 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") pod \"aff61f83-9ab8-403e-ad28-98afc53e8d00\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " Jun 10 15:07:07.487896 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.487783 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn844\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-kube-api-access-sn844\") pod \"aff61f83-9ab8-403e-ad28-98afc53e8d00\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " Jun 10 15:07:07.487896 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.487827 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-bound-sa-token\") pod \"aff61f83-9ab8-403e-ad28-98afc53e8d00\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " Jun 10 15:07:07.487896 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.487883 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff61f83-9ab8-403e-ad28-98afc53e8d00-ca-trust-extracted\") pod \"aff61f83-9ab8-403e-ad28-98afc53e8d00\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " Jun 10 15:07:07.488054 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.487921 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-installation-pull-secrets\") pod \"aff61f83-9ab8-403e-ad28-98afc53e8d00\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " Jun 10 15:07:07.488054 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.487990 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-trusted-ca\") pod \"aff61f83-9ab8-403e-ad28-98afc53e8d00\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " Jun 10 15:07:07.488054 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.488030 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-image-registry-private-configuration\") pod \"aff61f83-9ab8-403e-ad28-98afc53e8d00\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " Jun 10 15:07:07.488200 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.488056 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-certificates\") pod \"aff61f83-9ab8-403e-ad28-98afc53e8d00\" (UID: \"aff61f83-9ab8-403e-ad28-98afc53e8d00\") " Jun 10 15:07:07.488638 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.488611 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "aff61f83-9ab8-403e-ad28-98afc53e8d00" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:07:07.488923 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.488872 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "aff61f83-9ab8-403e-ad28-98afc53e8d00" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:07:07.490216 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.490184 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "aff61f83-9ab8-403e-ad28-98afc53e8d00" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:07:07.490556 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.490510 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "aff61f83-9ab8-403e-ad28-98afc53e8d00" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 15:07:07.490661 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.490583 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-kube-api-access-sn844" (OuterVolumeSpecName: "kube-api-access-sn844") pod "aff61f83-9ab8-403e-ad28-98afc53e8d00" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00"). InnerVolumeSpecName "kube-api-access-sn844". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:07:07.490735 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.490715 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "aff61f83-9ab8-403e-ad28-98afc53e8d00" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:07:07.490903 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.490879 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "aff61f83-9ab8-403e-ad28-98afc53e8d00" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 15:07:07.496348 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.496323 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff61f83-9ab8-403e-ad28-98afc53e8d00-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "aff61f83-9ab8-403e-ad28-98afc53e8d00" (UID: "aff61f83-9ab8-403e-ad28-98afc53e8d00"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:07:07.543373 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.543348 2561 generic.go:356] "Generic (PLEG): container finished" podID="aff61f83-9ab8-403e-ad28-98afc53e8d00" containerID="997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855" exitCode=0 Jun 10 15:07:07.543456 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.543383 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" Jun 10 15:07:07.543456 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.543416 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" event={"ID":"aff61f83-9ab8-403e-ad28-98afc53e8d00","Type":"ContainerDied","Data":"997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855"} Jun 10 15:07:07.543456 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.543452 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddf7f6777-mnb8x" event={"ID":"aff61f83-9ab8-403e-ad28-98afc53e8d00","Type":"ContainerDied","Data":"827361b2779159e137cc5ec56462a36ed09d02cbc780f67929ec68c99971fdf9"} Jun 10 15:07:07.543561 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.543471 2561 scope.go:117] "RemoveContainer" containerID="997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855" Jun 10 15:07:07.551366 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.551346 2561 scope.go:117] "RemoveContainer" containerID="997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855" Jun 10 15:07:07.551606 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:07:07.551585 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855\": container with ID starting with 997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855 not found: ID does not exist" containerID="997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855" Jun 10 15:07:07.551672 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.551617 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855"} err="failed to get container status \"997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855\": rpc error: code = NotFound desc = could not find container \"997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855\": container with ID starting with 997133b9e74bcb4ec4227dd39a52c49c5913124a2e018ad29c0cc3cab6077855 not found: ID does not exist" Jun 10 15:07:07.565401 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.565381 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ddf7f6777-mnb8x"] Jun 10 15:07:07.574809 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.574778 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-ddf7f6777-mnb8x"] Jun 10 15:07:07.589089 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.589065 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff61f83-9ab8-403e-ad28-98afc53e8d00-ca-trust-extracted\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:07.589089 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.589087 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-installation-pull-secrets\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:07.589224 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.589102 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-trusted-ca\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:07.589224 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.589116 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff61f83-9ab8-403e-ad28-98afc53e8d00-image-registry-private-configuration\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:07.589224 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.589130 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-certificates\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:07.589224 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.589143 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-registry-tls\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:07.589224 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.589157 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn844\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-kube-api-access-sn844\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:07.589224 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:07.589170 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff61f83-9ab8-403e-ad28-98afc53e8d00-bound-sa-token\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:08.047501 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:08.047469 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:08.126579 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:08.126555 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:09.342862 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:09.342818 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aff61f83-9ab8-403e-ad28-98afc53e8d00" path="/var/lib/kubelet/pods/aff61f83-9ab8-403e-ad28-98afc53e8d00/volumes" Jun 10 15:07:10.510480 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:10.510450 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:10.597112 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:10.597080 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:15.773538 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:15.773428 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-64cdd78d57-45tgx" podUID="ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" containerName="console" containerID="cri-o://9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e" gracePeriod=15 Jun 10 15:07:16.053068 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.053045 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:07:16.149322 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149293 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-service-ca\") pod \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " Jun 10 15:07:16.149440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149339 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-serving-cert\") pod \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " Jun 10 15:07:16.149440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149380 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twqqw\" (UniqueName: \"kubernetes.io/projected/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-kube-api-access-twqqw\") pod \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " Jun 10 15:07:16.149440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149397 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-oauth-serving-cert\") pod \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " Jun 10 15:07:16.149440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149414 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-oauth-config\") pod \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " Jun 10 15:07:16.149440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149435 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-trusted-ca-bundle\") pod \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " Jun 10 15:07:16.149688 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149461 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-config\") pod \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\" (UID: \"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79\") " Jun 10 15:07:16.149688 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149666 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-service-ca" (OuterVolumeSpecName: "service-ca") pod "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" (UID: "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:07:16.149948 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149906 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-config" (OuterVolumeSpecName: "console-config") pod "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" (UID: "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:07:16.150089 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.149958 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" (UID: "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:07:16.150089 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.150036 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" (UID: "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:07:16.151517 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.151495 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" (UID: "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 15:07:16.151517 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.151503 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-kube-api-access-twqqw" (OuterVolumeSpecName: "kube-api-access-twqqw") pod "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" (UID: "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79"). InnerVolumeSpecName "kube-api-access-twqqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:07:16.151641 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.151589 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" (UID: "ac6a9c01-e824-4c4e-8e83-8dc501fc2c79"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 15:07:16.250347 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.250314 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-service-ca\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:16.250347 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.250345 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-serving-cert\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:16.250484 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.250360 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twqqw\" (UniqueName: \"kubernetes.io/projected/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-kube-api-access-twqqw\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:16.250484 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.250372 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-oauth-serving-cert\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:16.250484 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.250384 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-oauth-config\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:16.250484 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.250396 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-trusted-ca-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:16.250484 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.250407 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79-console-config\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:07:16.577410 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.577385 2561 generic.go:356] "Generic (PLEG): container finished" podID="ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" containerID="9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e" exitCode=0 Jun 10 15:07:16.577515 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.577424 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cdd78d57-45tgx" Jun 10 15:07:16.577515 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.577480 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cdd78d57-45tgx" event={"ID":"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79","Type":"ContainerDied","Data":"9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e"} Jun 10 15:07:16.577628 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.577529 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cdd78d57-45tgx" event={"ID":"ac6a9c01-e824-4c4e-8e83-8dc501fc2c79","Type":"ContainerDied","Data":"37497a8765c72ecdabdf11eef8cc392e84df80b9df4fe8fbbc84ee5af1fcd181"} Jun 10 15:07:16.577628 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.577559 2561 scope.go:117] "RemoveContainer" containerID="9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e" Jun 10 15:07:16.585432 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.585414 2561 scope.go:117] "RemoveContainer" containerID="9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e" Jun 10 15:07:16.585671 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:07:16.585654 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e\": container with ID starting with 9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e not found: ID does not exist" containerID="9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e" Jun 10 15:07:16.585717 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.585680 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e"} err="failed to get container status \"9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e\": rpc error: code = NotFound desc = could not find container \"9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e\": container with ID starting with 9209e965c72eca6c60c7ba6703c45c855d3d612745bb3e55eedb91e3f9c2fd5e not found: ID does not exist" Jun 10 15:07:16.599779 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.599758 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64cdd78d57-45tgx"] Jun 10 15:07:16.604590 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:16.604571 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64cdd78d57-45tgx"] Jun 10 15:07:17.344395 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:17.344364 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" path="/var/lib/kubelet/pods/ac6a9c01-e824-4c4e-8e83-8dc501fc2c79/volumes" Jun 10 15:07:25.510374 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:25.510327 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:25.620607 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:25.620578 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:34.279257 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:34.279225 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:34.377825 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:34.377800 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:37.129118 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:37.129085 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:37.230947 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:37.230903 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:49.080768 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:49.080732 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:49.179833 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:49.179800 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:51.280238 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:51.280209 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:07:51.282408 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:51.282382 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce-metrics-certs\") pod \"network-metrics-daemon-msxf9\" (UID: \"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce\") " pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:07:51.544092 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:51.544075 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sz4k2\"" Jun 10 15:07:51.549396 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:51.549379 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msxf9" Jun 10 15:07:51.674585 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:51.674559 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-msxf9"] Jun 10 15:07:51.678216 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:07:51.678194 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef62f32_a211_4e9a_b7f5_e77c3dcbe4ce.slice/crio-058c9065c523ac0f9422d5ec551f27fc94b9ce3ac5a41f6b966bdbdc956c2c84 WatchSource:0}: Error finding container 058c9065c523ac0f9422d5ec551f27fc94b9ce3ac5a41f6b966bdbdc956c2c84: Status 404 returned error can't find the container with id 058c9065c523ac0f9422d5ec551f27fc94b9ce3ac5a41f6b966bdbdc956c2c84 Jun 10 15:07:51.680883 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:51.680868 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 15:07:51.699243 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:51.699220 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-msxf9" event={"ID":"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce","Type":"ContainerStarted","Data":"058c9065c523ac0f9422d5ec551f27fc94b9ce3ac5a41f6b966bdbdc956c2c84"} Jun 10 15:07:52.704736 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:52.704702 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-msxf9" event={"ID":"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce","Type":"ContainerStarted","Data":"0fdab8adadeeace29add6c0ed32ecd6d0abbca7ecd1dec811914ae335ee52f39"} Jun 10 15:07:53.709580 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:53.709539 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-msxf9" event={"ID":"6ef62f32-a211-4e9a-b7f5-e77c3dcbe4ce","Type":"ContainerStarted","Data":"2c53cff2d1bed70bc1dd7891b62b98cc98cb963ce26aabe380ea04e944646e29"} Jun 10 15:07:53.730367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:53.730318 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-msxf9" podStartSLOduration=375.820231005 podStartE2EDuration="6m16.730304119s" totalImagesPullingTime="910.073114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:07:53.728686266 +0000 UTC m=+376.860979070" watchObservedRunningTime="2026-06-10 15:07:53.730304119 +0000 UTC m=+376.862596912" Jun 10 15:07:53.841360 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:53.841331 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:53.942130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:53.942103 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:54.043098 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043067 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dfd6bf448-46mz4"] Jun 10 15:07:54.043440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043424 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54cc10b3-d8d6-4ab0-982d-85c5e92a22db" containerName="console" Jun 10 15:07:54.043440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043442 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="54cc10b3-d8d6-4ab0-982d-85c5e92a22db" containerName="console" Jun 10 15:07:54.043516 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043453 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff61f83-9ab8-403e-ad28-98afc53e8d00" containerName="registry" Jun 10 15:07:54.043516 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043459 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff61f83-9ab8-403e-ad28-98afc53e8d00" containerName="registry" Jun 10 15:07:54.043516 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043473 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" containerName="console" Jun 10 15:07:54.043516 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043480 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" containerName="console" Jun 10 15:07:54.043637 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043531 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="54cc10b3-d8d6-4ab0-982d-85c5e92a22db" containerName="console" Jun 10 15:07:54.043637 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043538 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac6a9c01-e824-4c4e-8e83-8dc501fc2c79" containerName="console" Jun 10 15:07:54.043637 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.043545 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff61f83-9ab8-403e-ad28-98afc53e8d00" containerName="registry" Jun 10 15:07:54.045938 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.045914 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.048815 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.048796 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 10 15:07:54.048950 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.048812 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 10 15:07:54.048950 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.048848 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f5cnw\"" Jun 10 15:07:54.049232 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.049216 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 10 15:07:54.049232 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.049226 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 10 15:07:54.049487 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.049471 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 10 15:07:54.057026 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.056953 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 10 15:07:54.058635 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.058615 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfd6bf448-46mz4"] Jun 10 15:07:54.115689 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.115654 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:54.200137 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.200101 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-serving-cert\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.200137 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.200140 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77bqk\" (UniqueName: \"kubernetes.io/projected/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-kube-api-access-77bqk\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.200349 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.200174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-oauth-config\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.200349 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.200197 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-service-ca\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.200349 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.200241 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-config\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.200349 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.200265 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-oauth-serving-cert\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.200349 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.200285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-trusted-ca-bundle\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.229365 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.229325 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:54.300689 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.300607 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-config\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.300689 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.300648 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-oauth-serving-cert\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.300853 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.300770 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-trusted-ca-bundle\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.300853 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.300831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-serving-cert\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.300947 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.300865 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77bqk\" (UniqueName: \"kubernetes.io/projected/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-kube-api-access-77bqk\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.300947 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.300908 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-oauth-config\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.301051 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.300957 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-service-ca\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.301497 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.301473 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-config\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.301497 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.301497 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-oauth-serving-cert\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.301707 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.301566 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-service-ca\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.301707 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.301625 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-trusted-ca-bundle\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.303284 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.303266 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-oauth-config\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.303430 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.303413 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-serving-cert\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.314412 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.314393 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77bqk\" (UniqueName: \"kubernetes.io/projected/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-kube-api-access-77bqk\") pod \"console-6dfd6bf448-46mz4\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.356790 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.356769 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:07:54.488364 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.488334 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfd6bf448-46mz4"] Jun 10 15:07:54.492389 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:07:54.492363 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1acddd3_68b4_48c1_8dfe_1e8217dfd039.slice/crio-2059f598cc476203b9a810f6b3803867e15248339d2a00a64323a84266f01b2e WatchSource:0}: Error finding container 2059f598cc476203b9a810f6b3803867e15248339d2a00a64323a84266f01b2e: Status 404 returned error can't find the container with id 2059f598cc476203b9a810f6b3803867e15248339d2a00a64323a84266f01b2e Jun 10 15:07:54.608549 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.608446 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:54.708794 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.708760 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:54.714041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.714014 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfd6bf448-46mz4" event={"ID":"a1acddd3-68b4-48c1-8dfe-1e8217dfd039","Type":"ContainerStarted","Data":"a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c"} Jun 10 15:07:54.714388 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.714057 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfd6bf448-46mz4" event={"ID":"a1acddd3-68b4-48c1-8dfe-1e8217dfd039","Type":"ContainerStarted","Data":"2059f598cc476203b9a810f6b3803867e15248339d2a00a64323a84266f01b2e"} Jun 10 15:07:54.734773 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:54.734730 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dfd6bf448-46mz4" podStartSLOduration=0.734715071 podStartE2EDuration="734.715071ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:07:54.734189248 +0000 UTC m=+377.866482043" watchObservedRunningTime="2026-06-10 15:07:54.734715071 +0000 UTC m=+377.867007866" Jun 10 15:07:55.515795 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:55.515764 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:55.614385 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:55.614352 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:07:55.694190 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:07:55.694156 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:08:04.357002 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:08:04.356967 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:08:04.357482 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:08:04.357015 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:08:04.361714 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:08:04.361690 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:08:04.758800 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:08:04.758715 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:08:05.011456 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:08:05.011378 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:08:05.109185 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:08:05.109155 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:08:17.007033 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:08:17.006989 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:08:17.115742 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:08:17.115711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:08:40.104847 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:08:40.104806 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:09:34.516562 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:34.516531 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:09:34.616030 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:34.615979 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:09:35.101996 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.101967 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk"] Jun 10 15:09:35.105098 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.105080 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.108814 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.108791 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nmjw\"" Jun 10 15:09:35.108902 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.108815 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 15:09:35.109118 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.109104 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 15:09:35.115832 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.115808 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk"] Jun 10 15:09:35.204717 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.204696 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.204825 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.204737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sp7f\" (UniqueName: \"kubernetes.io/projected/dfb922e5-9625-4f1e-9661-53579132aa49-kube-api-access-6sp7f\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.204825 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.204766 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.305884 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.305832 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.306077 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.305903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sp7f\" (UniqueName: \"kubernetes.io/projected/dfb922e5-9625-4f1e-9661-53579132aa49-kube-api-access-6sp7f\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.306077 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.305964 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.306261 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.306242 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.306303 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.306269 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.325792 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.325763 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sp7f\" (UniqueName: \"kubernetes.io/projected/dfb922e5-9625-4f1e-9661-53579132aa49-kube-api-access-6sp7f\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.414036 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.413957 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:09:35.544663 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:35.544640 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk"] Jun 10 15:09:35.546952 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:09:35.546900 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb922e5_9625_4f1e_9661_53579132aa49.slice/crio-1ffc9d6a349c9d4bc1028354c89a7245e28f454acf74e860a87ef2b00c97478d WatchSource:0}: Error finding container 1ffc9d6a349c9d4bc1028354c89a7245e28f454acf74e860a87ef2b00c97478d: Status 404 returned error can't find the container with id 1ffc9d6a349c9d4bc1028354c89a7245e28f454acf74e860a87ef2b00c97478d Jun 10 15:09:36.086861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:36.086832 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" event={"ID":"dfb922e5-9625-4f1e-9661-53579132aa49","Type":"ContainerStarted","Data":"1ffc9d6a349c9d4bc1028354c89a7245e28f454acf74e860a87ef2b00c97478d"} Jun 10 15:09:44.117864 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:44.117831 2561 generic.go:356] "Generic (PLEG): container finished" podID="dfb922e5-9625-4f1e-9661-53579132aa49" containerID="c65b5ef961e5f8174151e6a6a40368ca7da54d0f4fee83c1527983512045356f" exitCode=0 Jun 10 15:09:44.118247 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:44.117924 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" event={"ID":"dfb922e5-9625-4f1e-9661-53579132aa49","Type":"ContainerDied","Data":"c65b5ef961e5f8174151e6a6a40368ca7da54d0f4fee83c1527983512045356f"} Jun 10 15:09:50.143091 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:50.143061 2561 generic.go:356] "Generic (PLEG): container finished" podID="dfb922e5-9625-4f1e-9661-53579132aa49" containerID="2ac49a8de5e36f497b2286484d5fb561ca2ebb1dadc8ba4d0aa61fe1575286a1" exitCode=0 Jun 10 15:09:50.143446 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:09:50.143101 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" event={"ID":"dfb922e5-9625-4f1e-9661-53579132aa49","Type":"ContainerDied","Data":"2ac49a8de5e36f497b2286484d5fb561ca2ebb1dadc8ba4d0aa61fe1575286a1"} Jun 10 15:10:00.185545 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:00.185508 2561 generic.go:356] "Generic (PLEG): container finished" podID="dfb922e5-9625-4f1e-9661-53579132aa49" containerID="29cbd2eef429e33c324f0d2456c14d1863766bf8a9e92d742ff5bc687bd36d2a" exitCode=0 Jun 10 15:10:00.185888 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:00.185585 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" event={"ID":"dfb922e5-9625-4f1e-9661-53579132aa49","Type":"ContainerDied","Data":"29cbd2eef429e33c324f0d2456c14d1863766bf8a9e92d742ff5bc687bd36d2a"} Jun 10 15:10:01.344248 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.344225 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:10:01.422319 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.422295 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-bundle\") pod \"dfb922e5-9625-4f1e-9661-53579132aa49\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " Jun 10 15:10:01.422470 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.422373 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sp7f\" (UniqueName: \"kubernetes.io/projected/dfb922e5-9625-4f1e-9661-53579132aa49-kube-api-access-6sp7f\") pod \"dfb922e5-9625-4f1e-9661-53579132aa49\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " Jun 10 15:10:01.422470 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.422423 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-util\") pod \"dfb922e5-9625-4f1e-9661-53579132aa49\" (UID: \"dfb922e5-9625-4f1e-9661-53579132aa49\") " Jun 10 15:10:01.423045 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.423022 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-bundle" (OuterVolumeSpecName: "bundle") pod "dfb922e5-9625-4f1e-9661-53579132aa49" (UID: "dfb922e5-9625-4f1e-9661-53579132aa49"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:10:01.424499 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.424473 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfb922e5-9625-4f1e-9661-53579132aa49-kube-api-access-6sp7f" (OuterVolumeSpecName: "kube-api-access-6sp7f") pod "dfb922e5-9625-4f1e-9661-53579132aa49" (UID: "dfb922e5-9625-4f1e-9661-53579132aa49"). InnerVolumeSpecName "kube-api-access-6sp7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:10:01.427535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.427515 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-util" (OuterVolumeSpecName: "util") pod "dfb922e5-9625-4f1e-9661-53579132aa49" (UID: "dfb922e5-9625-4f1e-9661-53579132aa49"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:10:01.523999 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.523944 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:10:01.523999 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.523966 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfb922e5-9625-4f1e-9661-53579132aa49-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:10:01.523999 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:01.523977 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sp7f\" (UniqueName: \"kubernetes.io/projected/dfb922e5-9625-4f1e-9661-53579132aa49-kube-api-access-6sp7f\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:10:02.133440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:02.133405 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:02.194884 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:02.194851 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" event={"ID":"dfb922e5-9625-4f1e-9661-53579132aa49","Type":"ContainerDied","Data":"1ffc9d6a349c9d4bc1028354c89a7245e28f454acf74e860a87ef2b00c97478d"} Jun 10 15:10:02.194884 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:02.194869 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvpgk" Jun 10 15:10:02.195056 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:02.194896 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ffc9d6a349c9d4bc1028354c89a7245e28f454acf74e860a87ef2b00c97478d" Jun 10 15:10:06.268402 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:06.268370 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:07.147776 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:07.147741 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:08.276022 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.275990 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27"] Jun 10 15:10:08.276368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.276289 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfb922e5-9625-4f1e-9661-53579132aa49" containerName="util" Jun 10 15:10:08.276368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.276304 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfb922e5-9625-4f1e-9661-53579132aa49" containerName="util" Jun 10 15:10:08.276368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.276322 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfb922e5-9625-4f1e-9661-53579132aa49" containerName="pull" Jun 10 15:10:08.276368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.276327 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfb922e5-9625-4f1e-9661-53579132aa49" containerName="pull" Jun 10 15:10:08.276368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.276336 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfb922e5-9625-4f1e-9661-53579132aa49" containerName="extract" Jun 10 15:10:08.276368 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.276341 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfb922e5-9625-4f1e-9661-53579132aa49" containerName="extract" Jun 10 15:10:08.276553 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.276401 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfb922e5-9625-4f1e-9661-53579132aa49" containerName="extract" Jun 10 15:10:08.279161 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.279143 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" Jun 10 15:10:08.283155 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.283130 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 10 15:10:08.283278 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.283223 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 10 15:10:08.283331 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.283320 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-f78gs\"" Jun 10 15:10:08.298945 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.298910 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27"] Jun 10 15:10:08.371651 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.371624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jqm5\" (UniqueName: \"kubernetes.io/projected/d60d0172-4fa0-47cc-ae71-0ef017aa7920-kube-api-access-6jqm5\") pod \"cert-manager-operator-controller-manager-57757f7cfc-5wr27\" (UID: \"d60d0172-4fa0-47cc-ae71-0ef017aa7920\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" Jun 10 15:10:08.371809 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.371709 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d60d0172-4fa0-47cc-ae71-0ef017aa7920-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-5wr27\" (UID: \"d60d0172-4fa0-47cc-ae71-0ef017aa7920\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" Jun 10 15:10:08.472650 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.472625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d60d0172-4fa0-47cc-ae71-0ef017aa7920-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-5wr27\" (UID: \"d60d0172-4fa0-47cc-ae71-0ef017aa7920\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" Jun 10 15:10:08.472763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.472662 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jqm5\" (UniqueName: \"kubernetes.io/projected/d60d0172-4fa0-47cc-ae71-0ef017aa7920-kube-api-access-6jqm5\") pod \"cert-manager-operator-controller-manager-57757f7cfc-5wr27\" (UID: \"d60d0172-4fa0-47cc-ae71-0ef017aa7920\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" Jun 10 15:10:08.473050 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.473032 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d60d0172-4fa0-47cc-ae71-0ef017aa7920-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-5wr27\" (UID: \"d60d0172-4fa0-47cc-ae71-0ef017aa7920\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" Jun 10 15:10:08.490782 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.490763 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jqm5\" (UniqueName: \"kubernetes.io/projected/d60d0172-4fa0-47cc-ae71-0ef017aa7920-kube-api-access-6jqm5\") pod \"cert-manager-operator-controller-manager-57757f7cfc-5wr27\" (UID: \"d60d0172-4fa0-47cc-ae71-0ef017aa7920\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" Jun 10 15:10:08.587406 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.587384 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" Jun 10 15:10:08.740009 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:08.739977 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27"] Jun 10 15:10:08.743486 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:10:08.743458 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60d0172_4fa0_47cc_ae71_0ef017aa7920.slice/crio-c7b50a471e2c4ad31b859725e04819fa01e47d74d3a0fa3f0195ec909378197b WatchSource:0}: Error finding container c7b50a471e2c4ad31b859725e04819fa01e47d74d3a0fa3f0195ec909378197b: Status 404 returned error can't find the container with id c7b50a471e2c4ad31b859725e04819fa01e47d74d3a0fa3f0195ec909378197b Jun 10 15:10:09.227707 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:09.227677 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" event={"ID":"d60d0172-4fa0-47cc-ae71-0ef017aa7920","Type":"ContainerStarted","Data":"c7b50a471e2c4ad31b859725e04819fa01e47d74d3a0fa3f0195ec909378197b"} Jun 10 15:10:10.233414 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:10.233378 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" event={"ID":"d60d0172-4fa0-47cc-ae71-0ef017aa7920","Type":"ContainerStarted","Data":"3341b32b0beaf36b25990cfd1b8bd22b17b3825ad3c6397244782ef200cf8c10"} Jun 10 15:10:10.263951 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:10.263308 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-5wr27" podStartSLOduration=0.911265775 podStartE2EDuration="2.263290058s" totalImagesPullingTime="1.352024283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:10:10.261368845 +0000 UTC m=+513.393661638" watchObservedRunningTime="2026-06-10 15:10:10.263290058 +0000 UTC m=+513.395582853" Jun 10 15:10:10.279661 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:10.279624 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:10.399888 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:10.399809 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:10.659088 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:10.658984 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:10.784391 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:10.784355 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:11.010182 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:11.010097 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:11.139003 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:11.138969 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:12.062331 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.062298 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt"] Jun 10 15:10:12.065239 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.065221 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr"] Jun 10 15:10:12.065355 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.065342 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:12.067879 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.067864 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.069327 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.069296 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 10 15:10:12.069563 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.069549 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rvghs\"" Jun 10 15:10:12.069849 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.069833 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 10 15:10:12.070277 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.070263 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 15:10:12.070347 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.070322 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nmjw\"" Jun 10 15:10:12.070418 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.070390 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 15:10:12.102644 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.102623 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt"] Jun 10 15:10:12.103947 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.103912 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.104043 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.103990 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmfp7\" (UniqueName: \"kubernetes.io/projected/65a41b30-587e-4003-a3f1-ad6dee4f462e-kube-api-access-dmfp7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.104043 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.104025 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.104139 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.104050 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8pvs\" (UniqueName: \"kubernetes.io/projected/951bebfa-754d-43b7-a7ee-a286327d185a-kube-api-access-l8pvs\") pod \"cert-manager-webhook-7dc76cbb94-x8wlt\" (UID: \"951bebfa-754d-43b7-a7ee-a286327d185a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:12.104139 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.104106 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/951bebfa-754d-43b7-a7ee-a286327d185a-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-x8wlt\" (UID: \"951bebfa-754d-43b7-a7ee-a286327d185a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:12.106534 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.106510 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr"] Jun 10 15:10:12.205349 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.205319 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.205502 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.205354 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8pvs\" (UniqueName: \"kubernetes.io/projected/951bebfa-754d-43b7-a7ee-a286327d185a-kube-api-access-l8pvs\") pod \"cert-manager-webhook-7dc76cbb94-x8wlt\" (UID: \"951bebfa-754d-43b7-a7ee-a286327d185a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:12.205502 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.205376 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/951bebfa-754d-43b7-a7ee-a286327d185a-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-x8wlt\" (UID: \"951bebfa-754d-43b7-a7ee-a286327d185a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:12.205502 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.205406 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.205502 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.205442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmfp7\" (UniqueName: \"kubernetes.io/projected/65a41b30-587e-4003-a3f1-ad6dee4f462e-kube-api-access-dmfp7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.205784 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.205765 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.205843 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.205788 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.224898 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.224864 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/951bebfa-754d-43b7-a7ee-a286327d185a-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-x8wlt\" (UID: \"951bebfa-754d-43b7-a7ee-a286327d185a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:12.225870 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.225851 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmfp7\" (UniqueName: \"kubernetes.io/projected/65a41b30-587e-4003-a3f1-ad6dee4f462e-kube-api-access-dmfp7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.248415 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.248386 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8pvs\" (UniqueName: \"kubernetes.io/projected/951bebfa-754d-43b7-a7ee-a286327d185a-kube-api-access-l8pvs\") pod \"cert-manager-webhook-7dc76cbb94-x8wlt\" (UID: \"951bebfa-754d-43b7-a7ee-a286327d185a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:12.389539 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.389453 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:12.394040 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.394017 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:12.582513 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.582486 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr"] Jun 10 15:10:12.585763 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:10:12.585737 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a41b30_587e_4003_a3f1_ad6dee4f462e.slice/crio-0f8930894239896f7c040de5aa075200477179ce43133c7d67e6370d63b01577 WatchSource:0}: Error finding container 0f8930894239896f7c040de5aa075200477179ce43133c7d67e6370d63b01577: Status 404 returned error can't find the container with id 0f8930894239896f7c040de5aa075200477179ce43133c7d67e6370d63b01577 Jun 10 15:10:12.599144 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:12.599121 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt"] Jun 10 15:10:12.599575 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:10:12.599551 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod951bebfa_754d_43b7_a7ee_a286327d185a.slice/crio-052388d8fbc94b07800276692b9a3175f5eccfd852be72b89073cec36629d9dd WatchSource:0}: Error finding container 052388d8fbc94b07800276692b9a3175f5eccfd852be72b89073cec36629d9dd: Status 404 returned error can't find the container with id 052388d8fbc94b07800276692b9a3175f5eccfd852be72b89073cec36629d9dd Jun 10 15:10:13.246612 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:13.246578 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" event={"ID":"951bebfa-754d-43b7-a7ee-a286327d185a","Type":"ContainerStarted","Data":"052388d8fbc94b07800276692b9a3175f5eccfd852be72b89073cec36629d9dd"} Jun 10 15:10:13.248079 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:13.248054 2561 generic.go:356] "Generic (PLEG): container finished" podID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerID="ff7e1c9dd515e243beb8f823816a77e10d0c8e6badc9f82f37c75d11564863c9" exitCode=0 Jun 10 15:10:13.248195 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:13.248112 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" event={"ID":"65a41b30-587e-4003-a3f1-ad6dee4f462e","Type":"ContainerDied","Data":"ff7e1c9dd515e243beb8f823816a77e10d0c8e6badc9f82f37c75d11564863c9"} Jun 10 15:10:13.248195 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:13.248157 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" event={"ID":"65a41b30-587e-4003-a3f1-ad6dee4f462e","Type":"ContainerStarted","Data":"0f8930894239896f7c040de5aa075200477179ce43133c7d67e6370d63b01577"} Jun 10 15:10:14.414209 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:14.414174 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:14.613737 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:14.613702 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:17.271190 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:17.271156 2561 generic.go:356] "Generic (PLEG): container finished" podID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerID="3e3f764d0f8fa8f77bea3396ce35de8805c2cc210d412556eed90e80688ab0bd" exitCode=0 Jun 10 15:10:17.271637 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:17.271254 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" event={"ID":"65a41b30-587e-4003-a3f1-ad6dee4f462e","Type":"ContainerDied","Data":"3e3f764d0f8fa8f77bea3396ce35de8805c2cc210d412556eed90e80688ab0bd"} Jun 10 15:10:17.272711 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:17.272683 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" event={"ID":"951bebfa-754d-43b7-a7ee-a286327d185a","Type":"ContainerStarted","Data":"980edd65fe68909effc4ab4b0a9e96ac917ddc9420ce070ee71c39a4ca8fe8d5"} Jun 10 15:10:17.272840 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:17.272818 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:18.279157 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:18.279126 2561 generic.go:356] "Generic (PLEG): container finished" podID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerID="6d70fa455b3494b5c8b60e41638c114a2c411bd6537798c0fb33b0f04b1e3c29" exitCode=0 Jun 10 15:10:18.279538 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:18.279219 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" event={"ID":"65a41b30-587e-4003-a3f1-ad6dee4f462e","Type":"ContainerDied","Data":"6d70fa455b3494b5c8b60e41638c114a2c411bd6537798c0fb33b0f04b1e3c29"} Jun 10 15:10:18.319472 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:18.319419 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" podStartSLOduration=2.403520318 podStartE2EDuration="6.319402821s" totalImagesPullingTime="3.915882503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:10:17.351870461 +0000 UTC m=+520.484163251" watchObservedRunningTime="2026-06-10 15:10:18.319402821 +0000 UTC m=+521.451695619" Jun 10 15:10:19.412427 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.412405 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:19.468097 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.468070 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmfp7\" (UniqueName: \"kubernetes.io/projected/65a41b30-587e-4003-a3f1-ad6dee4f462e-kube-api-access-dmfp7\") pod \"65a41b30-587e-4003-a3f1-ad6dee4f462e\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " Jun 10 15:10:19.468231 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.468124 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-bundle\") pod \"65a41b30-587e-4003-a3f1-ad6dee4f462e\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " Jun 10 15:10:19.468231 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.468151 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-util\") pod \"65a41b30-587e-4003-a3f1-ad6dee4f462e\" (UID: \"65a41b30-587e-4003-a3f1-ad6dee4f462e\") " Jun 10 15:10:19.468489 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.468465 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-bundle" (OuterVolumeSpecName: "bundle") pod "65a41b30-587e-4003-a3f1-ad6dee4f462e" (UID: "65a41b30-587e-4003-a3f1-ad6dee4f462e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:10:19.470168 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.470137 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65a41b30-587e-4003-a3f1-ad6dee4f462e-kube-api-access-dmfp7" (OuterVolumeSpecName: "kube-api-access-dmfp7") pod "65a41b30-587e-4003-a3f1-ad6dee4f462e" (UID: "65a41b30-587e-4003-a3f1-ad6dee4f462e"). InnerVolumeSpecName "kube-api-access-dmfp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:10:19.472799 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.472761 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-util" (OuterVolumeSpecName: "util") pod "65a41b30-587e-4003-a3f1-ad6dee4f462e" (UID: "65a41b30-587e-4003-a3f1-ad6dee4f462e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:10:19.569584 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.569561 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmfp7\" (UniqueName: \"kubernetes.io/projected/65a41b30-587e-4003-a3f1-ad6dee4f462e-kube-api-access-dmfp7\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:10:19.569584 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.569584 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:10:19.569698 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:19.569593 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65a41b30-587e-4003-a3f1-ad6dee4f462e-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:10:20.294156 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:20.294110 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" event={"ID":"65a41b30-587e-4003-a3f1-ad6dee4f462e","Type":"ContainerDied","Data":"0f8930894239896f7c040de5aa075200477179ce43133c7d67e6370d63b01577"} Jun 10 15:10:20.294156 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:20.294147 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f45djr" Jun 10 15:10:20.294388 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:20.294208 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f8930894239896f7c040de5aa075200477179ce43133c7d67e6370d63b01577" Jun 10 15:10:20.690333 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:20.690251 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:20.803195 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:20.803163 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:21.591768 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:21.591743 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:23.151553 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:23.151520 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:23.267893 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:23.267860 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:23.282139 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:23.282108 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-x8wlt" Jun 10 15:10:26.363180 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.363136 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-5fqdr"] Jun 10 15:10:26.363571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.363461 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerName="util" Jun 10 15:10:26.363571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.363475 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerName="util" Jun 10 15:10:26.363571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.363491 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerName="pull" Jun 10 15:10:26.363571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.363497 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerName="pull" Jun 10 15:10:26.363571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.363505 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerName="extract" Jun 10 15:10:26.363571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.363510 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerName="extract" Jun 10 15:10:26.363571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.363559 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="65a41b30-587e-4003-a3f1-ad6dee4f462e" containerName="extract" Jun 10 15:10:26.366162 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.366143 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-5fqdr" Jun 10 15:10:26.368784 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.368759 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-rp6tg\"" Jun 10 15:10:26.401264 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.401238 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-5fqdr"] Jun 10 15:10:26.422778 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.422753 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffcbae56-44e6-4f70-8107-0ed1a6b66236-bound-sa-token\") pod \"cert-manager-59c77684cc-5fqdr\" (UID: \"ffcbae56-44e6-4f70-8107-0ed1a6b66236\") " pod="cert-manager/cert-manager-59c77684cc-5fqdr" Jun 10 15:10:26.422914 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.422896 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8csj\" (UniqueName: \"kubernetes.io/projected/ffcbae56-44e6-4f70-8107-0ed1a6b66236-kube-api-access-v8csj\") pod \"cert-manager-59c77684cc-5fqdr\" (UID: \"ffcbae56-44e6-4f70-8107-0ed1a6b66236\") " pod="cert-manager/cert-manager-59c77684cc-5fqdr" Jun 10 15:10:26.523581 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.523557 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8csj\" (UniqueName: \"kubernetes.io/projected/ffcbae56-44e6-4f70-8107-0ed1a6b66236-kube-api-access-v8csj\") pod \"cert-manager-59c77684cc-5fqdr\" (UID: \"ffcbae56-44e6-4f70-8107-0ed1a6b66236\") " pod="cert-manager/cert-manager-59c77684cc-5fqdr" Jun 10 15:10:26.523669 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.523623 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffcbae56-44e6-4f70-8107-0ed1a6b66236-bound-sa-token\") pod \"cert-manager-59c77684cc-5fqdr\" (UID: \"ffcbae56-44e6-4f70-8107-0ed1a6b66236\") " pod="cert-manager/cert-manager-59c77684cc-5fqdr" Jun 10 15:10:26.535757 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.535727 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8csj\" (UniqueName: \"kubernetes.io/projected/ffcbae56-44e6-4f70-8107-0ed1a6b66236-kube-api-access-v8csj\") pod \"cert-manager-59c77684cc-5fqdr\" (UID: \"ffcbae56-44e6-4f70-8107-0ed1a6b66236\") " pod="cert-manager/cert-manager-59c77684cc-5fqdr" Jun 10 15:10:26.536860 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.536842 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffcbae56-44e6-4f70-8107-0ed1a6b66236-bound-sa-token\") pod \"cert-manager-59c77684cc-5fqdr\" (UID: \"ffcbae56-44e6-4f70-8107-0ed1a6b66236\") " pod="cert-manager/cert-manager-59c77684cc-5fqdr" Jun 10 15:10:26.676520 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.676455 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-5fqdr" Jun 10 15:10:26.804238 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:26.804202 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-5fqdr"] Jun 10 15:10:26.826111 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:10:26.826086 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffcbae56_44e6_4f70_8107_0ed1a6b66236.slice/crio-6a161a22a3fc7797f15c21ac9a9f24f549819d6fc1544b9ddec6dc7d3d210755 WatchSource:0}: Error finding container 6a161a22a3fc7797f15c21ac9a9f24f549819d6fc1544b9ddec6dc7d3d210755: Status 404 returned error can't find the container with id 6a161a22a3fc7797f15c21ac9a9f24f549819d6fc1544b9ddec6dc7d3d210755 Jun 10 15:10:27.331721 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:27.331688 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-5fqdr" event={"ID":"ffcbae56-44e6-4f70-8107-0ed1a6b66236","Type":"ContainerStarted","Data":"135ef5a208b5db9bba14948dae7d57eeb8285e4a2c6f715f07962713d40e97b9"} Jun 10 15:10:27.331873 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:27.331736 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-5fqdr" event={"ID":"ffcbae56-44e6-4f70-8107-0ed1a6b66236","Type":"ContainerStarted","Data":"6a161a22a3fc7797f15c21ac9a9f24f549819d6fc1544b9ddec6dc7d3d210755"} Jun 10 15:10:27.354439 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:27.354389 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-5fqdr" podStartSLOduration=1.354374377 podStartE2EDuration="1.354374377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:10:27.350973177 +0000 UTC m=+530.483265965" watchObservedRunningTime="2026-06-10 15:10:27.354374377 +0000 UTC m=+530.486667167" Jun 10 15:10:29.923365 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:29.923333 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:30.040037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:30.040011 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:31.751553 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.751513 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2"] Jun 10 15:10:31.797176 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.797146 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2"] Jun 10 15:10:31.797322 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.797222 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.801422 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.801403 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 15:10:31.802466 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.802444 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nmjw\"" Jun 10 15:10:31.802581 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.802510 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 15:10:31.861319 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.861299 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.861451 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.861331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz9t5\" (UniqueName: \"kubernetes.io/projected/cdefe490-6aea-49a4-9728-ffc00f01e7d8-kube-api-access-kz9t5\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.861451 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.861364 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.961706 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.961684 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.961809 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.961712 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz9t5\" (UniqueName: \"kubernetes.io/projected/cdefe490-6aea-49a4-9728-ffc00f01e7d8-kube-api-access-kz9t5\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.961809 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.961737 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.962139 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.962121 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.962175 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.962135 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.981823 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.981797 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz9t5\" (UniqueName: \"kubernetes.io/projected/cdefe490-6aea-49a4-9728-ffc00f01e7d8-kube-api-access-kz9t5\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:31.990781 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:31.990761 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:32.107006 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:32.106985 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:32.264336 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:32.264305 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2"] Jun 10 15:10:32.268437 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:10:32.268413 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdefe490_6aea_49a4_9728_ffc00f01e7d8.slice/crio-8d579e0c05139f3adac9e7f2377bbcb4ba0f44b1c460c574c955923004e3f2be WatchSource:0}: Error finding container 8d579e0c05139f3adac9e7f2377bbcb4ba0f44b1c460c574c955923004e3f2be: Status 404 returned error can't find the container with id 8d579e0c05139f3adac9e7f2377bbcb4ba0f44b1c460c574c955923004e3f2be Jun 10 15:10:32.355601 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:32.355576 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" event={"ID":"cdefe490-6aea-49a4-9728-ffc00f01e7d8","Type":"ContainerStarted","Data":"fb34ce9b4e090c55ef49189f971f05ea798c63658b75d21cfddc45a27518108d"} Jun 10 15:10:32.355686 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:32.355620 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" event={"ID":"cdefe490-6aea-49a4-9728-ffc00f01e7d8","Type":"ContainerStarted","Data":"8d579e0c05139f3adac9e7f2377bbcb4ba0f44b1c460c574c955923004e3f2be"} Jun 10 15:10:33.361488 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:33.361455 2561 generic.go:356] "Generic (PLEG): container finished" podID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerID="fb34ce9b4e090c55ef49189f971f05ea798c63658b75d21cfddc45a27518108d" exitCode=0 Jun 10 15:10:33.361912 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:33.361505 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" event={"ID":"cdefe490-6aea-49a4-9728-ffc00f01e7d8","Type":"ContainerDied","Data":"fb34ce9b4e090c55ef49189f971f05ea798c63658b75d21cfddc45a27518108d"} Jun 10 15:10:34.367333 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:34.367301 2561 generic.go:356] "Generic (PLEG): container finished" podID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerID="48eaa70189fd300cbafab31e28a959d61e2651022712db322f273cb38c1f6861" exitCode=0 Jun 10 15:10:34.367631 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:34.367379 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" event={"ID":"cdefe490-6aea-49a4-9728-ffc00f01e7d8","Type":"ContainerDied","Data":"48eaa70189fd300cbafab31e28a959d61e2651022712db322f273cb38c1f6861"} Jun 10 15:10:35.373586 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:35.373552 2561 generic.go:356] "Generic (PLEG): container finished" podID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerID="641505f6e0c999903776f7b9ceeb967057f47a0c196fb27b725db7e6119daafb" exitCode=0 Jun 10 15:10:35.374010 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:35.373618 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" event={"ID":"cdefe490-6aea-49a4-9728-ffc00f01e7d8","Type":"ContainerDied","Data":"641505f6e0c999903776f7b9ceeb967057f47a0c196fb27b725db7e6119daafb"} Jun 10 15:10:36.506969 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.506947 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:36.599942 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.599908 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-bundle\") pod \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " Jun 10 15:10:36.600056 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.599975 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz9t5\" (UniqueName: \"kubernetes.io/projected/cdefe490-6aea-49a4-9728-ffc00f01e7d8-kube-api-access-kz9t5\") pod \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " Jun 10 15:10:36.600056 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.600032 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-util\") pod \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\" (UID: \"cdefe490-6aea-49a4-9728-ffc00f01e7d8\") " Jun 10 15:10:36.600826 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.600779 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-bundle" (OuterVolumeSpecName: "bundle") pod "cdefe490-6aea-49a4-9728-ffc00f01e7d8" (UID: "cdefe490-6aea-49a4-9728-ffc00f01e7d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:10:36.601990 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.601961 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdefe490-6aea-49a4-9728-ffc00f01e7d8-kube-api-access-kz9t5" (OuterVolumeSpecName: "kube-api-access-kz9t5") pod "cdefe490-6aea-49a4-9728-ffc00f01e7d8" (UID: "cdefe490-6aea-49a4-9728-ffc00f01e7d8"). InnerVolumeSpecName "kube-api-access-kz9t5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:10:36.605856 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.605828 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-util" (OuterVolumeSpecName: "util") pod "cdefe490-6aea-49a4-9728-ffc00f01e7d8" (UID: "cdefe490-6aea-49a4-9728-ffc00f01e7d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:10:36.700944 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.700878 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:10:36.700944 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.700900 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz9t5\" (UniqueName: \"kubernetes.io/projected/cdefe490-6aea-49a4-9728-ffc00f01e7d8-kube-api-access-kz9t5\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:10:36.700944 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:36.700910 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdefe490-6aea-49a4-9728-ffc00f01e7d8-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:10:37.383877 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:37.383857 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" Jun 10 15:10:37.384006 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:37.383873 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd46dw2" event={"ID":"cdefe490-6aea-49a4-9728-ffc00f01e7d8","Type":"ContainerDied","Data":"8d579e0c05139f3adac9e7f2377bbcb4ba0f44b1c460c574c955923004e3f2be"} Jun 10 15:10:37.384006 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:37.383911 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d579e0c05139f3adac9e7f2377bbcb4ba0f44b1c460c574c955923004e3f2be" Jun 10 15:10:43.446437 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:43.446408 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:43.561367 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:43.561333 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:44.028095 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.028065 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:44.337960 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.337916 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:44.449092 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.449066 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:44.517535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.517508 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct"] Jun 10 15:10:44.517803 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.517790 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerName="pull" Jun 10 15:10:44.517803 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.517803 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerName="pull" Jun 10 15:10:44.517882 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.517826 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerName="util" Jun 10 15:10:44.517882 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.517832 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerName="util" Jun 10 15:10:44.517882 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.517838 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerName="extract" Jun 10 15:10:44.517882 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.517843 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerName="extract" Jun 10 15:10:44.518054 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.517903 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdefe490-6aea-49a4-9728-ffc00f01e7d8" containerName="extract" Jun 10 15:10:44.521007 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.520992 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.524274 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.524255 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 10 15:10:44.524393 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.524302 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 10 15:10:44.524903 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.524887 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 10 15:10:44.528878 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.528853 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 10 15:10:44.537239 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.537219 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jmg8q\"" Jun 10 15:10:44.548674 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.548656 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct"] Jun 10 15:10:44.651571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.651514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61585677-1cfc-4feb-802c-010c1d873c2a-tmp\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.651571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.651545 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hj57\" (UniqueName: \"kubernetes.io/projected/61585677-1cfc-4feb-802c-010c1d873c2a-kube-api-access-9hj57\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.651571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.651569 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61585677-1cfc-4feb-802c-010c1d873c2a-webhook-cert\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.651737 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.651596 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61585677-1cfc-4feb-802c-010c1d873c2a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.651737 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.651647 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61585677-1cfc-4feb-802c-010c1d873c2a-manifests\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.703772 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.703743 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:44.752095 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.752069 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61585677-1cfc-4feb-802c-010c1d873c2a-tmp\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.752202 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.752098 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hj57\" (UniqueName: \"kubernetes.io/projected/61585677-1cfc-4feb-802c-010c1d873c2a-kube-api-access-9hj57\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.752202 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.752120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61585677-1cfc-4feb-802c-010c1d873c2a-webhook-cert\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.752202 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.752148 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61585677-1cfc-4feb-802c-010c1d873c2a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.752202 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.752189 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61585677-1cfc-4feb-802c-010c1d873c2a-manifests\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.752716 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.752690 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61585677-1cfc-4feb-802c-010c1d873c2a-manifests\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.754285 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.754260 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61585677-1cfc-4feb-802c-010c1d873c2a-tmp\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.754414 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.754396 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61585677-1cfc-4feb-802c-010c1d873c2a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.754496 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.754480 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61585677-1cfc-4feb-802c-010c1d873c2a-webhook-cert\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.761861 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.761839 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hj57\" (UniqueName: \"kubernetes.io/projected/61585677-1cfc-4feb-802c-010c1d873c2a-kube-api-access-9hj57\") pod \"opendatahub-operator-controller-manager-58876c898f-mh7ct\" (UID: \"61585677-1cfc-4feb-802c-010c1d873c2a\") " pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.814717 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.814689 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:44.830864 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.830843 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:44.920794 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.920766 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:44.963763 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:44.963741 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct"] Jun 10 15:10:44.966337 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:10:44.966312 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61585677_1cfc_4feb_802c_010c1d873c2a.slice/crio-2b4843998a19bea5a83cb2403a52ef4cece5bd80abea088c1dfbbd55efd4b8bd WatchSource:0}: Error finding container 2b4843998a19bea5a83cb2403a52ef4cece5bd80abea088c1dfbbd55efd4b8bd: Status 404 returned error can't find the container with id 2b4843998a19bea5a83cb2403a52ef4cece5bd80abea088c1dfbbd55efd4b8bd Jun 10 15:10:45.024903 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:45.024874 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:45.117455 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:45.117430 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:45.217820 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:45.217760 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:45.318110 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:45.318090 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:45.422621 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:45.422599 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" event={"ID":"61585677-1cfc-4feb-802c-010c1d873c2a","Type":"ContainerStarted","Data":"2b4843998a19bea5a83cb2403a52ef4cece5bd80abea088c1dfbbd55efd4b8bd"} Jun 10 15:10:48.440546 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:48.440517 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" event={"ID":"61585677-1cfc-4feb-802c-010c1d873c2a","Type":"ContainerStarted","Data":"d2ec0cd29378d704789b79f3090db509843ef909c549767997130ea809f44c5e"} Jun 10 15:10:49.446503 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:49.446469 2561 generic.go:356] "Generic (PLEG): container finished" podID="61585677-1cfc-4feb-802c-010c1d873c2a" containerID="d2ec0cd29378d704789b79f3090db509843ef909c549767997130ea809f44c5e" exitCode=0 Jun 10 15:10:49.446941 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:49.446553 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" event={"ID":"61585677-1cfc-4feb-802c-010c1d873c2a","Type":"ContainerDied","Data":"d2ec0cd29378d704789b79f3090db509843ef909c549767997130ea809f44c5e"} Jun 10 15:10:50.452590 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:50.452535 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" event={"ID":"61585677-1cfc-4feb-802c-010c1d873c2a","Type":"ContainerStarted","Data":"9c75047f00cced19f7072f9c3cd74afd8dd56075a5bd66e4fbaf4496da605f6b"} Jun 10 15:10:50.453018 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:50.452659 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:10:50.477696 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:50.477655 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" podStartSLOduration=2.091312472 podStartE2EDuration="6.477641386s" totalImagesPullingTime="3.426850639s" totalInitContainerRuntime="959.478275ms" isStatefulPod=false podCreationTimestamp="2026-06-10 15:10:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:10:50.476298295 +0000 UTC m=+553.608591090" watchObservedRunningTime="2026-06-10 15:10:50.477641386 +0000 UTC m=+553.609934179" Jun 10 15:10:53.941051 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:53.941005 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:54.070614 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:54.070584 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:10:59.052217 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:10:59.052186 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:01.459534 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:01.459503 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58876c898f-mh7ct" Jun 10 15:11:04.614187 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:04.614160 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:04.751818 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:04.751793 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:05.468375 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.468339 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-659rm"] Jun 10 15:11:05.477147 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.477129 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:05.481270 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.481253 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 10 15:11:05.489513 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.489494 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-r9mkj\"" Jun 10 15:11:05.492511 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.492491 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-659rm"] Jun 10 15:11:05.497868 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.497843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85893829-8aa2-4182-9726-0a6b4a3cef3f-cert\") pod \"odh-model-controller-6dc94f689f-659rm\" (UID: \"85893829-8aa2-4182-9726-0a6b4a3cef3f\") " pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:05.497997 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.497906 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blr5h\" (UniqueName: \"kubernetes.io/projected/85893829-8aa2-4182-9726-0a6b4a3cef3f-kube-api-access-blr5h\") pod \"odh-model-controller-6dc94f689f-659rm\" (UID: \"85893829-8aa2-4182-9726-0a6b4a3cef3f\") " pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:05.598317 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.598297 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85893829-8aa2-4182-9726-0a6b4a3cef3f-cert\") pod \"odh-model-controller-6dc94f689f-659rm\" (UID: \"85893829-8aa2-4182-9726-0a6b4a3cef3f\") " pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:05.598417 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.598338 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blr5h\" (UniqueName: \"kubernetes.io/projected/85893829-8aa2-4182-9726-0a6b4a3cef3f-kube-api-access-blr5h\") pod \"odh-model-controller-6dc94f689f-659rm\" (UID: \"85893829-8aa2-4182-9726-0a6b4a3cef3f\") " pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:05.598483 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.598460 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:05.600769 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.600741 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85893829-8aa2-4182-9726-0a6b4a3cef3f-cert\") pod \"odh-model-controller-6dc94f689f-659rm\" (UID: \"85893829-8aa2-4182-9726-0a6b4a3cef3f\") " pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:05.615176 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.615158 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blr5h\" (UniqueName: \"kubernetes.io/projected/85893829-8aa2-4182-9726-0a6b4a3cef3f-kube-api-access-blr5h\") pod \"odh-model-controller-6dc94f689f-659rm\" (UID: \"85893829-8aa2-4182-9726-0a6b4a3cef3f\") " pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:05.779035 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.778978 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:05.786903 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.786883 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:05.903195 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.903164 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:05.938250 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:05.938206 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-659rm"] Jun 10 15:11:05.942007 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:11:05.941978 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85893829_8aa2_4182_9726_0a6b4a3cef3f.slice/crio-1b6ecbc0f0cf3b87e4c80578bc12e10d4d095e1de3e57172da26049939366079 WatchSource:0}: Error finding container 1b6ecbc0f0cf3b87e4c80578bc12e10d4d095e1de3e57172da26049939366079: Status 404 returned error can't find the container with id 1b6ecbc0f0cf3b87e4c80578bc12e10d4d095e1de3e57172da26049939366079 Jun 10 15:11:06.312935 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.312904 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:06.479942 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.479907 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:06.531913 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.531878 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-659rm" event={"ID":"85893829-8aa2-4182-9726-0a6b4a3cef3f","Type":"ContainerStarted","Data":"1b6ecbc0f0cf3b87e4c80578bc12e10d4d095e1de3e57172da26049939366079"} Jun 10 15:11:06.600451 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.600374 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-ppxll"] Jun 10 15:11:06.603782 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.603759 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:06.606741 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.606710 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghtv6\" (UniqueName: \"kubernetes.io/projected/e4e30e07-7b00-40cb-99d3-e429f480fd68-kube-api-access-ghtv6\") pod \"model-serving-api-5dd488fc48-ppxll\" (UID: \"e4e30e07-7b00-40cb-99d3-e429f480fd68\") " pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:06.606859 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.606810 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4e30e07-7b00-40cb-99d3-e429f480fd68-tls-certs\") pod \"model-serving-api-5dd488fc48-ppxll\" (UID: \"e4e30e07-7b00-40cb-99d3-e429f480fd68\") " pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:06.613006 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.612855 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 10 15:11:06.627893 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.627871 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-ppxll"] Jun 10 15:11:06.668295 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.668268 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:06.707300 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.707276 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4e30e07-7b00-40cb-99d3-e429f480fd68-tls-certs\") pod \"model-serving-api-5dd488fc48-ppxll\" (UID: \"e4e30e07-7b00-40cb-99d3-e429f480fd68\") " pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:06.707401 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.707352 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghtv6\" (UniqueName: \"kubernetes.io/projected/e4e30e07-7b00-40cb-99d3-e429f480fd68-kube-api-access-ghtv6\") pod \"model-serving-api-5dd488fc48-ppxll\" (UID: \"e4e30e07-7b00-40cb-99d3-e429f480fd68\") " pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:06.707464 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:06.707431 2561 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 10 15:11:06.707519 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:06.707496 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e30e07-7b00-40cb-99d3-e429f480fd68-tls-certs podName:e4e30e07-7b00-40cb-99d3-e429f480fd68 nodeName:}" failed. No retries permitted until 2026-06-10 15:11:07.207475044 +0000 UTC m=+570.339767820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e4e30e07-7b00-40cb-99d3-e429f480fd68-tls-certs") pod "model-serving-api-5dd488fc48-ppxll" (UID: "e4e30e07-7b00-40cb-99d3-e429f480fd68") : secret "model-serving-api-tls" not found Jun 10 15:11:06.727697 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:06.727668 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghtv6\" (UniqueName: \"kubernetes.io/projected/e4e30e07-7b00-40cb-99d3-e429f480fd68-kube-api-access-ghtv6\") pod \"model-serving-api-5dd488fc48-ppxll\" (UID: \"e4e30e07-7b00-40cb-99d3-e429f480fd68\") " pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:07.165557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:07.165523 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:07.209768 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:07.209737 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4e30e07-7b00-40cb-99d3-e429f480fd68-tls-certs\") pod \"model-serving-api-5dd488fc48-ppxll\" (UID: \"e4e30e07-7b00-40cb-99d3-e429f480fd68\") " pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:07.212370 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:07.212350 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4e30e07-7b00-40cb-99d3-e429f480fd68-tls-certs\") pod \"model-serving-api-5dd488fc48-ppxll\" (UID: \"e4e30e07-7b00-40cb-99d3-e429f480fd68\") " pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:07.217124 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:07.217074 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:07.366945 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:07.366907 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-ppxll"] Jun 10 15:11:07.368697 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:11:07.368673 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e30e07_7b00_40cb_99d3_e429f480fd68.slice/crio-50071353ee076095a194b062adb9aefd1c49c952594f5879fb9d943cd85496e6 WatchSource:0}: Error finding container 50071353ee076095a194b062adb9aefd1c49c952594f5879fb9d943cd85496e6: Status 404 returned error can't find the container with id 50071353ee076095a194b062adb9aefd1c49c952594f5879fb9d943cd85496e6 Jun 10 15:11:07.537433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:07.537359 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-ppxll" event={"ID":"e4e30e07-7b00-40cb-99d3-e429f480fd68","Type":"ContainerStarted","Data":"50071353ee076095a194b062adb9aefd1c49c952594f5879fb9d943cd85496e6"} Jun 10 15:11:08.290199 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:08.290167 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:08.403415 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:08.403385 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:09.549272 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:09.549238 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-659rm" event={"ID":"85893829-8aa2-4182-9726-0a6b4a3cef3f","Type":"ContainerStarted","Data":"d8823278ca462ba36536cad63727af972605aae5af97a6ad2ebbaf879f05fff3"} Jun 10 15:11:09.549705 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:09.549313 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:09.572256 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:09.572167 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-659rm" podStartSLOduration=1.211004299 podStartE2EDuration="4.572148981s" totalImagesPullingTime="3.361144682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:11:09.570318273 +0000 UTC m=+572.702611068" watchObservedRunningTime="2026-06-10 15:11:09.572148981 +0000 UTC m=+572.704441775" Jun 10 15:11:10.133683 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:10.133640 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:10.227056 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:10.227003 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:11.559966 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:11.559921 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-ppxll" event={"ID":"e4e30e07-7b00-40cb-99d3-e429f480fd68","Type":"ContainerStarted","Data":"8f5eaf24b092599ced54aae831df851ba3a48bdc5b647cca5defe833e455c8fe"} Jun 10 15:11:11.560349 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:11.560010 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:11.609100 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:11.609052 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-ppxll" podStartSLOduration=2.391797417 podStartE2EDuration="5.609039023s" totalImagesPullingTime="3.217241606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:11:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:11:11.607912546 +0000 UTC m=+574.740205342" watchObservedRunningTime="2026-06-10 15:11:11.609039023 +0000 UTC m=+574.741331816" Jun 10 15:11:15.735171 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.735139 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:15.851398 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.851368 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b997c8484-lbblx"] Jun 10 15:11:15.854135 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.854119 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:15.856755 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.856731 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:15.869774 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.869759 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-86sdk\"" Jun 10 15:11:15.873017 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.872996 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 10 15:11:15.874466 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.874450 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9257b5dc-9c61-4254-abca-f1650c82ee87-tls-certs\") pod \"kube-auth-proxy-5b997c8484-lbblx\" (UID: \"9257b5dc-9c61-4254-abca-f1650c82ee87\") " pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:15.874515 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.874497 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p9l6\" (UniqueName: \"kubernetes.io/projected/9257b5dc-9c61-4254-abca-f1650c82ee87-kube-api-access-7p9l6\") pod \"kube-auth-proxy-5b997c8484-lbblx\" (UID: \"9257b5dc-9c61-4254-abca-f1650c82ee87\") " pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:15.874570 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.874532 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9257b5dc-9c61-4254-abca-f1650c82ee87-tmp\") pod \"kube-auth-proxy-5b997c8484-lbblx\" (UID: \"9257b5dc-9c61-4254-abca-f1650c82ee87\") " pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:15.880833 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.880818 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 10 15:11:15.903316 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.903295 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b997c8484-lbblx"] Jun 10 15:11:15.975185 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.975160 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9257b5dc-9c61-4254-abca-f1650c82ee87-tls-certs\") pod \"kube-auth-proxy-5b997c8484-lbblx\" (UID: \"9257b5dc-9c61-4254-abca-f1650c82ee87\") " pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:15.975276 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.975233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p9l6\" (UniqueName: \"kubernetes.io/projected/9257b5dc-9c61-4254-abca-f1650c82ee87-kube-api-access-7p9l6\") pod \"kube-auth-proxy-5b997c8484-lbblx\" (UID: \"9257b5dc-9c61-4254-abca-f1650c82ee87\") " pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:15.975369 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.975273 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9257b5dc-9c61-4254-abca-f1650c82ee87-tmp\") pod \"kube-auth-proxy-5b997c8484-lbblx\" (UID: \"9257b5dc-9c61-4254-abca-f1650c82ee87\") " pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:15.977290 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.977271 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9257b5dc-9c61-4254-abca-f1650c82ee87-tmp\") pod \"kube-auth-proxy-5b997c8484-lbblx\" (UID: \"9257b5dc-9c61-4254-abca-f1650c82ee87\") " pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:15.977428 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.977410 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9257b5dc-9c61-4254-abca-f1650c82ee87-tls-certs\") pod \"kube-auth-proxy-5b997c8484-lbblx\" (UID: \"9257b5dc-9c61-4254-abca-f1650c82ee87\") " pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:15.989902 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:15.989854 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p9l6\" (UniqueName: \"kubernetes.io/projected/9257b5dc-9c61-4254-abca-f1650c82ee87-kube-api-access-7p9l6\") pod \"kube-auth-proxy-5b997c8484-lbblx\" (UID: \"9257b5dc-9c61-4254-abca-f1650c82ee87\") " pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:16.018449 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.018427 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:16.080665 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.080640 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc"] Jun 10 15:11:16.083410 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.083396 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.086516 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.086496 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 15:11:16.086815 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.086801 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 15:11:16.091771 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.091756 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nmjw\"" Jun 10 15:11:16.096253 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.096233 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc"] Jun 10 15:11:16.163401 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.163372 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" Jun 10 15:11:16.177138 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.177111 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.177249 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.177166 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv48j\" (UniqueName: \"kubernetes.io/projected/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-kube-api-access-cv48j\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.177249 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.177213 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.212501 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.212463 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:16.278655 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.278536 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.279900 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.278860 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv48j\" (UniqueName: \"kubernetes.io/projected/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-kube-api-access-cv48j\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.279900 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.278954 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.279900 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.278984 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.279900 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.279838 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.303487 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.303463 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv48j\" (UniqueName: \"kubernetes.io/projected/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-kube-api-access-cv48j\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.331581 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.331533 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd"] Jun 10 15:11:16.335999 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.335979 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:16.336166 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.335983 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-5z7pm"] Jun 10 15:11:16.339285 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.339264 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:16.340182 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.340162 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5m75n\"" Jun 10 15:11:16.341899 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.341875 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 10 15:11:16.343271 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.342765 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:16.343271 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.342977 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b997c8484-lbblx"] Jun 10 15:11:16.343271 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.343133 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-z8gjh\"" Jun 10 15:11:16.343481 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.343356 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 10 15:11:16.344288 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:11:16.344264 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9257b5dc_9c61_4254_abca_f1650c82ee87.slice/crio-43163e860dce79b304cb40706010ec2bb32cfb75f19d53ac3a70abfd662ebaf0 WatchSource:0}: Error finding container 43163e860dce79b304cb40706010ec2bb32cfb75f19d53ac3a70abfd662ebaf0: Status 404 returned error can't find the container with id 43163e860dce79b304cb40706010ec2bb32cfb75f19d53ac3a70abfd662ebaf0 Jun 10 15:11:16.353345 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.353326 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd"] Jun 10 15:11:16.355439 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.355421 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-5z7pm"] Jun 10 15:11:16.380131 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.379542 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0d33a59-7ce9-4757-adf6-2271f8262bfc-cert\") pod \"kserve-controller-manager-69fc866d76-5z7pm\" (UID: \"a0d33a59-7ce9-4757-adf6-2271f8262bfc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:16.380131 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.379583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jk9g\" (UniqueName: \"kubernetes.io/projected/a0d33a59-7ce9-4757-adf6-2271f8262bfc-kube-api-access-4jk9g\") pod \"kserve-controller-manager-69fc866d76-5z7pm\" (UID: \"a0d33a59-7ce9-4757-adf6-2271f8262bfc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:16.380131 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.379624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4449h\" (UniqueName: \"kubernetes.io/projected/e0158ddf-8fcb-4088-b35e-85167eba1630-kube-api-access-4449h\") pod \"llmisvc-controller-manager-864bfd7bc9-zsddd\" (UID: \"e0158ddf-8fcb-4088-b35e-85167eba1630\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:16.380131 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.379661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0158ddf-8fcb-4088-b35e-85167eba1630-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-zsddd\" (UID: \"e0158ddf-8fcb-4088-b35e-85167eba1630\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:16.392263 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.392244 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:16.480988 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.480652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0d33a59-7ce9-4757-adf6-2271f8262bfc-cert\") pod \"kserve-controller-manager-69fc866d76-5z7pm\" (UID: \"a0d33a59-7ce9-4757-adf6-2271f8262bfc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:16.480988 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.480686 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jk9g\" (UniqueName: \"kubernetes.io/projected/a0d33a59-7ce9-4757-adf6-2271f8262bfc-kube-api-access-4jk9g\") pod \"kserve-controller-manager-69fc866d76-5z7pm\" (UID: \"a0d33a59-7ce9-4757-adf6-2271f8262bfc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:16.480988 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.480718 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4449h\" (UniqueName: \"kubernetes.io/projected/e0158ddf-8fcb-4088-b35e-85167eba1630-kube-api-access-4449h\") pod \"llmisvc-controller-manager-864bfd7bc9-zsddd\" (UID: \"e0158ddf-8fcb-4088-b35e-85167eba1630\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:16.480988 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.480756 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0158ddf-8fcb-4088-b35e-85167eba1630-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-zsddd\" (UID: \"e0158ddf-8fcb-4088-b35e-85167eba1630\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:16.480988 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:16.480819 2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 10 15:11:16.480988 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:16.480847 2561 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 10 15:11:16.480988 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:16.480890 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d33a59-7ce9-4757-adf6-2271f8262bfc-cert podName:a0d33a59-7ce9-4757-adf6-2271f8262bfc nodeName:}" failed. No retries permitted until 2026-06-10 15:11:16.980867281 +0000 UTC m=+580.113160060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a0d33a59-7ce9-4757-adf6-2271f8262bfc-cert") pod "kserve-controller-manager-69fc866d76-5z7pm" (UID: "a0d33a59-7ce9-4757-adf6-2271f8262bfc") : secret "kserve-webhook-server-cert" not found Jun 10 15:11:16.480988 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:16.480908 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0158ddf-8fcb-4088-b35e-85167eba1630-cert podName:e0158ddf-8fcb-4088-b35e-85167eba1630 nodeName:}" failed. No retries permitted until 2026-06-10 15:11:16.980898625 +0000 UTC m=+580.113191404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e0158ddf-8fcb-4088-b35e-85167eba1630-cert") pod "llmisvc-controller-manager-864bfd7bc9-zsddd" (UID: "e0158ddf-8fcb-4088-b35e-85167eba1630") : secret "llmisvc-webhook-server-cert" not found Jun 10 15:11:16.497966 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.497923 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4449h\" (UniqueName: \"kubernetes.io/projected/e0158ddf-8fcb-4088-b35e-85167eba1630-kube-api-access-4449h\") pod \"llmisvc-controller-manager-864bfd7bc9-zsddd\" (UID: \"e0158ddf-8fcb-4088-b35e-85167eba1630\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:16.499010 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.498982 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jk9g\" (UniqueName: \"kubernetes.io/projected/a0d33a59-7ce9-4757-adf6-2271f8262bfc-kube-api-access-4jk9g\") pod \"kserve-controller-manager-69fc866d76-5z7pm\" (UID: \"a0d33a59-7ce9-4757-adf6-2271f8262bfc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:16.544698 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.544668 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:16.546682 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.546658 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc"] Jun 10 15:11:16.548444 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:11:16.548422 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f6d625_c9aa_4cbb_b959_127e9d1e550c.slice/crio-d31baeb45483bcf4ef24804bc86f1bb9846dbdd886eb9daef0dac8e53bf79b0c WatchSource:0}: Error finding container d31baeb45483bcf4ef24804bc86f1bb9846dbdd886eb9daef0dac8e53bf79b0c: Status 404 returned error can't find the container with id d31baeb45483bcf4ef24804bc86f1bb9846dbdd886eb9daef0dac8e53bf79b0c Jun 10 15:11:16.584147 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.584119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" event={"ID":"c4f6d625-c9aa-4cbb-b959-127e9d1e550c","Type":"ContainerStarted","Data":"d31baeb45483bcf4ef24804bc86f1bb9846dbdd886eb9daef0dac8e53bf79b0c"} Jun 10 15:11:16.585271 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.585253 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" event={"ID":"9257b5dc-9c61-4254-abca-f1650c82ee87","Type":"ContainerStarted","Data":"43163e860dce79b304cb40706010ec2bb32cfb75f19d53ac3a70abfd662ebaf0"} Jun 10 15:11:16.647180 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.647083 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:16.986079 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.986002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0158ddf-8fcb-4088-b35e-85167eba1630-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-zsddd\" (UID: \"e0158ddf-8fcb-4088-b35e-85167eba1630\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:16.986079 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:16.986065 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0d33a59-7ce9-4757-adf6-2271f8262bfc-cert\") pod \"kserve-controller-manager-69fc866d76-5z7pm\" (UID: \"a0d33a59-7ce9-4757-adf6-2271f8262bfc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:16.986529 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:16.986172 2561 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 10 15:11:16.986529 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:16.986243 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0158ddf-8fcb-4088-b35e-85167eba1630-cert podName:e0158ddf-8fcb-4088-b35e-85167eba1630 nodeName:}" failed. No retries permitted until 2026-06-10 15:11:17.98622367 +0000 UTC m=+581.118516445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e0158ddf-8fcb-4088-b35e-85167eba1630-cert") pod "llmisvc-controller-manager-864bfd7bc9-zsddd" (UID: "e0158ddf-8fcb-4088-b35e-85167eba1630") : secret "llmisvc-webhook-server-cert" not found Jun 10 15:11:16.986529 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:16.986180 2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 10 15:11:16.986529 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:11:16.986320 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0d33a59-7ce9-4757-adf6-2271f8262bfc-cert podName:a0d33a59-7ce9-4757-adf6-2271f8262bfc nodeName:}" failed. No retries permitted until 2026-06-10 15:11:17.986303999 +0000 UTC m=+581.118596773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a0d33a59-7ce9-4757-adf6-2271f8262bfc-cert") pod "kserve-controller-manager-69fc866d76-5z7pm" (UID: "a0d33a59-7ce9-4757-adf6-2271f8262bfc") : secret "kserve-webhook-server-cert" not found Jun 10 15:11:17.210009 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:17.209979 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:17.477441 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:17.477352 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:17.590924 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:17.590892 2561 generic.go:356] "Generic (PLEG): container finished" podID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerID="9eb6f3f36c0058e0a89fd0a4f38900abf7754edf274219d3f5e15420809cdba2" exitCode=0 Jun 10 15:11:17.591102 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:17.590946 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" event={"ID":"c4f6d625-c9aa-4cbb-b959-127e9d1e550c","Type":"ContainerDied","Data":"9eb6f3f36c0058e0a89fd0a4f38900abf7754edf274219d3f5e15420809cdba2"} Jun 10 15:11:17.993839 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:17.993807 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0d33a59-7ce9-4757-adf6-2271f8262bfc-cert\") pod \"kserve-controller-manager-69fc866d76-5z7pm\" (UID: \"a0d33a59-7ce9-4757-adf6-2271f8262bfc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:17.994246 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:17.993868 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0158ddf-8fcb-4088-b35e-85167eba1630-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-zsddd\" (UID: \"e0158ddf-8fcb-4088-b35e-85167eba1630\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:17.996535 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:17.996514 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0158ddf-8fcb-4088-b35e-85167eba1630-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-zsddd\" (UID: \"e0158ddf-8fcb-4088-b35e-85167eba1630\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:17.996644 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:17.996581 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0d33a59-7ce9-4757-adf6-2271f8262bfc-cert\") pod \"kserve-controller-manager-69fc866d76-5z7pm\" (UID: \"a0d33a59-7ce9-4757-adf6-2271f8262bfc\") " pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:18.151678 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:18.151650 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:18.161783 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:18.161760 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:18.380428 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:11:18.380398 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0d33a59_7ce9_4757_adf6_2271f8262bfc.slice/crio-63811a5d171a810528833397149256473b3c5e0cf6d9e6935ae0afb7918831b1 WatchSource:0}: Error finding container 63811a5d171a810528833397149256473b3c5e0cf6d9e6935ae0afb7918831b1: Status 404 returned error can't find the container with id 63811a5d171a810528833397149256473b3c5e0cf6d9e6935ae0afb7918831b1 Jun 10 15:11:18.381539 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:18.381513 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-5z7pm"] Jun 10 15:11:18.408458 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:18.408437 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd"] Jun 10 15:11:18.411937 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:11:18.411899 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode0158ddf_8fcb_4088_b35e_85167eba1630.slice/crio-59f534b3e05ae33180089a549b055bdb4c1431b595defb0d5a96d8bfe82c7cf7 WatchSource:0}: Error finding container 59f534b3e05ae33180089a549b055bdb4c1431b595defb0d5a96d8bfe82c7cf7: Status 404 returned error can't find the container with id 59f534b3e05ae33180089a549b055bdb4c1431b595defb0d5a96d8bfe82c7cf7 Jun 10 15:11:18.597684 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:18.597649 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" event={"ID":"a0d33a59-7ce9-4757-adf6-2271f8262bfc","Type":"ContainerStarted","Data":"63811a5d171a810528833397149256473b3c5e0cf6d9e6935ae0afb7918831b1"} Jun 10 15:11:18.598945 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:18.598900 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" event={"ID":"e0158ddf-8fcb-4088-b35e-85167eba1630","Type":"ContainerStarted","Data":"59f534b3e05ae33180089a549b055bdb4c1431b595defb0d5a96d8bfe82c7cf7"} Jun 10 15:11:20.556686 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:20.556655 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-659rm" Jun 10 15:11:21.621805 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:21.621764 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" event={"ID":"9257b5dc-9c61-4254-abca-f1650c82ee87","Type":"ContainerStarted","Data":"a8c4c3a838817fba0e64f29dea018add9e0259e62ef4ae467495915b12a5f46d"} Jun 10 15:11:21.645427 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:21.645379 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b997c8484-lbblx" podStartSLOduration=3.207126954 podStartE2EDuration="6.645362252s" totalImagesPullingTime="3.438235298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:11:21.642974949 +0000 UTC m=+584.775267743" watchObservedRunningTime="2026-06-10 15:11:21.645362252 +0000 UTC m=+584.777655046" Jun 10 15:11:22.568001 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:22.567968 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-ppxll" Jun 10 15:11:22.630394 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:22.630352 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" event={"ID":"a0d33a59-7ce9-4757-adf6-2271f8262bfc","Type":"ContainerStarted","Data":"4f08164e0594121b1a9ad2b937de9922544b87584b8ffb6272c3abda4f0b5246"} Jun 10 15:11:22.650961 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:22.650896 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" podStartSLOduration=3.297411441 podStartE2EDuration="6.650879843s" totalImagesPullingTime="3.353468402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:11:22.650081295 +0000 UTC m=+585.782374090" watchObservedRunningTime="2026-06-10 15:11:22.650879843 +0000 UTC m=+585.783172637" Jun 10 15:11:23.636738 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:23.636706 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" event={"ID":"e0158ddf-8fcb-4088-b35e-85167eba1630","Type":"ContainerStarted","Data":"9eade4d4b82e739cf02f53c40f1a0dc1d7f27483afcf019b245c7d42b5cff556"} Jun 10 15:11:23.637201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:23.636984 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:23.637201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:23.637024 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:23.670604 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:23.670542 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" podStartSLOduration=2.827425839 podStartE2EDuration="7.670523795s" totalImagesPullingTime="4.843097956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:11:23.669773944 +0000 UTC m=+586.802066739" watchObservedRunningTime="2026-06-10 15:11:23.670523795 +0000 UTC m=+586.802816590" Jun 10 15:11:32.681441 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:32.681409 2561 generic.go:356] "Generic (PLEG): container finished" podID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerID="daed2d02fb9fb93d415aa2c912ab438fa4da611d5d3d02f1876e8fc74184f83c" exitCode=0 Jun 10 15:11:32.681814 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:32.681509 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" event={"ID":"c4f6d625-c9aa-4cbb-b959-127e9d1e550c","Type":"ContainerDied","Data":"daed2d02fb9fb93d415aa2c912ab438fa4da611d5d3d02f1876e8fc74184f83c"} Jun 10 15:11:33.688574 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:33.688535 2561 generic.go:356] "Generic (PLEG): container finished" podID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerID="0de9ad5542d0e2f1681964101e2defd2942db3b15eb5e9a8f7086f2cf80cd2f1" exitCode=0 Jun 10 15:11:33.689020 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:33.688629 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" event={"ID":"c4f6d625-c9aa-4cbb-b959-127e9d1e550c","Type":"ContainerDied","Data":"0de9ad5542d0e2f1681964101e2defd2942db3b15eb5e9a8f7086f2cf80cd2f1"} Jun 10 15:11:34.833514 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:34.833492 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:34.940596 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:34.940572 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-bundle\") pod \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " Jun 10 15:11:34.940707 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:34.940609 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-util\") pod \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " Jun 10 15:11:34.940707 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:34.940671 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv48j\" (UniqueName: \"kubernetes.io/projected/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-kube-api-access-cv48j\") pod \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\" (UID: \"c4f6d625-c9aa-4cbb-b959-127e9d1e550c\") " Jun 10 15:11:34.941435 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:34.941411 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-bundle" (OuterVolumeSpecName: "bundle") pod "c4f6d625-c9aa-4cbb-b959-127e9d1e550c" (UID: "c4f6d625-c9aa-4cbb-b959-127e9d1e550c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:11:34.942671 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:34.942650 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-kube-api-access-cv48j" (OuterVolumeSpecName: "kube-api-access-cv48j") pod "c4f6d625-c9aa-4cbb-b959-127e9d1e550c" (UID: "c4f6d625-c9aa-4cbb-b959-127e9d1e550c"). InnerVolumeSpecName "kube-api-access-cv48j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:11:34.945413 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:34.945370 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-util" (OuterVolumeSpecName: "util") pod "c4f6d625-c9aa-4cbb-b959-127e9d1e550c" (UID: "c4f6d625-c9aa-4cbb-b959-127e9d1e550c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:11:35.041777 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:35.041755 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv48j\" (UniqueName: \"kubernetes.io/projected/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-kube-api-access-cv48j\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:11:35.041777 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:35.041776 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:11:35.041891 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:35.041785 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4f6d625-c9aa-4cbb-b959-127e9d1e550c-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:11:35.701620 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:35.701552 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" event={"ID":"c4f6d625-c9aa-4cbb-b959-127e9d1e550c","Type":"ContainerDied","Data":"d31baeb45483bcf4ef24804bc86f1bb9846dbdd886eb9daef0dac8e53bf79b0c"} Jun 10 15:11:35.701620 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:35.701593 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab069bbkc" Jun 10 15:11:35.701620 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:35.701599 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d31baeb45483bcf4ef24804bc86f1bb9846dbdd886eb9daef0dac8e53bf79b0c" Jun 10 15:11:39.094126 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:39.094093 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:39.205036 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:39.205008 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:40.486040 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.486001 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv"] Jun 10 15:11:40.486534 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.486514 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerName="extract" Jun 10 15:11:40.486613 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.486536 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerName="extract" Jun 10 15:11:40.486613 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.486563 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerName="util" Jun 10 15:11:40.486613 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.486572 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerName="util" Jun 10 15:11:40.486613 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.486587 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerName="pull" Jun 10 15:11:40.486613 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.486596 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerName="pull" Jun 10 15:11:40.486869 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.486712 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4f6d625-c9aa-4cbb-b959-127e9d1e550c" containerName="extract" Jun 10 15:11:40.489272 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.489251 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.494796 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.494772 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 15:11:40.494894 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.494775 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nmjw\"" Jun 10 15:11:40.495993 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.495883 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 15:11:40.515202 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.515177 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv"] Jun 10 15:11:40.591439 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.591417 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.591542 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.591475 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cmjv\" (UniqueName: \"kubernetes.io/projected/f72df458-7f9f-4c0a-9389-c9b3424cee39-kube-api-access-7cmjv\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.591598 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.591583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.692022 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.691995 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.692114 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.692026 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.692182 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.692162 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cmjv\" (UniqueName: \"kubernetes.io/projected/f72df458-7f9f-4c0a-9389-c9b3424cee39-kube-api-access-7cmjv\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.692382 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.692367 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.692427 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.692392 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.705456 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.705430 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cmjv\" (UniqueName: \"kubernetes.io/projected/f72df458-7f9f-4c0a-9389-c9b3424cee39-kube-api-access-7cmjv\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.726382 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.726361 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:40.798243 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.798209 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:40.833860 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.833838 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:40.970134 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.970100 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:40.989653 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:40.989626 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv"] Jun 10 15:11:40.993528 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:11:40.993502 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf72df458_7f9f_4c0a_9389_c9b3424cee39.slice/crio-308db872b2ad069a9db7f22c7a40dd3ae1374a49384c17d0c02b2cdda7a71bdf WatchSource:0}: Error finding container 308db872b2ad069a9db7f22c7a40dd3ae1374a49384c17d0c02b2cdda7a71bdf: Status 404 returned error can't find the container with id 308db872b2ad069a9db7f22c7a40dd3ae1374a49384c17d0c02b2cdda7a71bdf Jun 10 15:11:41.582864 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:41.582827 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:41.685475 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:41.685442 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:41.732356 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:41.732326 2561 generic.go:356] "Generic (PLEG): container finished" podID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerID="b995b591079f903a1e3a61fe906916591f29c58823c9e97be1b77a8bed242f7d" exitCode=0 Jun 10 15:11:41.732470 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:41.732410 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" event={"ID":"f72df458-7f9f-4c0a-9389-c9b3424cee39","Type":"ContainerDied","Data":"b995b591079f903a1e3a61fe906916591f29c58823c9e97be1b77a8bed242f7d"} Jun 10 15:11:41.732470 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:41.732450 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" event={"ID":"f72df458-7f9f-4c0a-9389-c9b3424cee39","Type":"ContainerStarted","Data":"308db872b2ad069a9db7f22c7a40dd3ae1374a49384c17d0c02b2cdda7a71bdf"} Jun 10 15:11:42.232496 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.232464 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk"] Jun 10 15:11:42.245856 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.245829 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:42.249364 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.249341 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 10 15:11:42.249530 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.249474 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-v97nl\"" Jun 10 15:11:42.249775 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.249758 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 10 15:11:42.266793 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.266763 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk"] Jun 10 15:11:42.407729 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.407701 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/2e8ca265-d5d8-43ad-9fce-f07ff1aff584-operator-config\") pod \"servicemesh-operator3-57f65f65fb-mvqsk\" (UID: \"2e8ca265-d5d8-43ad-9fce-f07ff1aff584\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:42.407729 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.407733 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-764t4\" (UniqueName: \"kubernetes.io/projected/2e8ca265-d5d8-43ad-9fce-f07ff1aff584-kube-api-access-764t4\") pod \"servicemesh-operator3-57f65f65fb-mvqsk\" (UID: \"2e8ca265-d5d8-43ad-9fce-f07ff1aff584\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:42.509090 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.509004 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/2e8ca265-d5d8-43ad-9fce-f07ff1aff584-operator-config\") pod \"servicemesh-operator3-57f65f65fb-mvqsk\" (UID: \"2e8ca265-d5d8-43ad-9fce-f07ff1aff584\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:42.509321 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.509300 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-764t4\" (UniqueName: \"kubernetes.io/projected/2e8ca265-d5d8-43ad-9fce-f07ff1aff584-kube-api-access-764t4\") pod \"servicemesh-operator3-57f65f65fb-mvqsk\" (UID: \"2e8ca265-d5d8-43ad-9fce-f07ff1aff584\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:42.513291 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.513261 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/2e8ca265-d5d8-43ad-9fce-f07ff1aff584-operator-config\") pod \"servicemesh-operator3-57f65f65fb-mvqsk\" (UID: \"2e8ca265-d5d8-43ad-9fce-f07ff1aff584\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:42.524826 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.524802 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-764t4\" (UniqueName: \"kubernetes.io/projected/2e8ca265-d5d8-43ad-9fce-f07ff1aff584-kube-api-access-764t4\") pod \"servicemesh-operator3-57f65f65fb-mvqsk\" (UID: \"2e8ca265-d5d8-43ad-9fce-f07ff1aff584\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:42.555823 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.555791 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:42.713887 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.713857 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk"] Jun 10 15:11:42.716175 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:11:42.716143 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8ca265_d5d8_43ad_9fce_f07ff1aff584.slice/crio-5148caddaf772ad90a55390fa114b40e03894629ccf10983d3c8e8cd58e59e37 WatchSource:0}: Error finding container 5148caddaf772ad90a55390fa114b40e03894629ccf10983d3c8e8cd58e59e37: Status 404 returned error can't find the container with id 5148caddaf772ad90a55390fa114b40e03894629ccf10983d3c8e8cd58e59e37 Jun 10 15:11:42.739216 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:42.739182 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" event={"ID":"2e8ca265-d5d8-43ad-9fce-f07ff1aff584","Type":"ContainerStarted","Data":"5148caddaf772ad90a55390fa114b40e03894629ccf10983d3c8e8cd58e59e37"} Jun 10 15:11:43.746982 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:43.746955 2561 generic.go:356] "Generic (PLEG): container finished" podID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerID="fdb14166b58c4888c202556d05404410d266c04f5a1f505e926d46ef5253772d" exitCode=0 Jun 10 15:11:43.747419 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:43.747044 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" event={"ID":"f72df458-7f9f-4c0a-9389-c9b3424cee39","Type":"ContainerDied","Data":"fdb14166b58c4888c202556d05404410d266c04f5a1f505e926d46ef5253772d"} Jun 10 15:11:44.755298 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:44.755265 2561 generic.go:356] "Generic (PLEG): container finished" podID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerID="22d149fd3456eb364235ee539164d614b7a0aa5cac046902fd5240157afadcea" exitCode=0 Jun 10 15:11:44.755715 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:44.755337 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" event={"ID":"f72df458-7f9f-4c0a-9389-c9b3424cee39","Type":"ContainerDied","Data":"22d149fd3456eb364235ee539164d614b7a0aa5cac046902fd5240157afadcea"} Jun 10 15:11:45.762620 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:45.762581 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" event={"ID":"2e8ca265-d5d8-43ad-9fce-f07ff1aff584","Type":"ContainerStarted","Data":"ccc276bf8b6f5fcb7f3cb115d80599d3bb6c5c7515fadae18ff758311bbb8c6c"} Jun 10 15:11:45.763023 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:45.762677 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:45.790200 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:45.790154 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" podStartSLOduration=0.85823651 podStartE2EDuration="3.790137789s" totalImagesPullingTime="2.931901279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:11:45.787665982 +0000 UTC m=+608.919958777" watchObservedRunningTime="2026-06-10 15:11:45.790137789 +0000 UTC m=+608.922430583" Jun 10 15:11:45.917947 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:45.917903 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:46.039660 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.039625 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-bundle\") pod \"f72df458-7f9f-4c0a-9389-c9b3424cee39\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " Jun 10 15:11:46.039830 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.039676 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cmjv\" (UniqueName: \"kubernetes.io/projected/f72df458-7f9f-4c0a-9389-c9b3424cee39-kube-api-access-7cmjv\") pod \"f72df458-7f9f-4c0a-9389-c9b3424cee39\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " Jun 10 15:11:46.039830 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.039725 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-util\") pod \"f72df458-7f9f-4c0a-9389-c9b3424cee39\" (UID: \"f72df458-7f9f-4c0a-9389-c9b3424cee39\") " Jun 10 15:11:46.040464 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.040435 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-bundle" (OuterVolumeSpecName: "bundle") pod "f72df458-7f9f-4c0a-9389-c9b3424cee39" (UID: "f72df458-7f9f-4c0a-9389-c9b3424cee39"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:11:46.041780 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.041756 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f72df458-7f9f-4c0a-9389-c9b3424cee39-kube-api-access-7cmjv" (OuterVolumeSpecName: "kube-api-access-7cmjv") pod "f72df458-7f9f-4c0a-9389-c9b3424cee39" (UID: "f72df458-7f9f-4c0a-9389-c9b3424cee39"). InnerVolumeSpecName "kube-api-access-7cmjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:11:46.045083 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.045051 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-util" (OuterVolumeSpecName: "util") pod "f72df458-7f9f-4c0a-9389-c9b3424cee39" (UID: "f72df458-7f9f-4c0a-9389-c9b3424cee39"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:11:46.141169 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.141145 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:11:46.141169 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.141167 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cmjv\" (UniqueName: \"kubernetes.io/projected/f72df458-7f9f-4c0a-9389-c9b3424cee39-kube-api-access-7cmjv\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:11:46.141289 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.141177 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f72df458-7f9f-4c0a-9389-c9b3424cee39-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:11:46.768953 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.768899 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" event={"ID":"f72df458-7f9f-4c0a-9389-c9b3424cee39","Type":"ContainerDied","Data":"308db872b2ad069a9db7f22c7a40dd3ae1374a49384c17d0c02b2cdda7a71bdf"} Jun 10 15:11:46.768953 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.768922 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762afmrlv" Jun 10 15:11:46.769387 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:46.768971 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="308db872b2ad069a9db7f22c7a40dd3ae1374a49384c17d0c02b2cdda7a71bdf" Jun 10 15:11:52.035176 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.035136 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx"] Jun 10 15:11:52.035781 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.035738 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerName="pull" Jun 10 15:11:52.035781 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.035759 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerName="pull" Jun 10 15:11:52.035781 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.035770 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerName="extract" Jun 10 15:11:52.035781 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.035778 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerName="extract" Jun 10 15:11:52.036031 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.035796 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerName="util" Jun 10 15:11:52.036031 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.035804 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerName="util" Jun 10 15:11:52.036031 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.035893 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f72df458-7f9f-4c0a-9389-c9b3424cee39" containerName="extract" Jun 10 15:11:52.040738 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.040714 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.045264 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.045234 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 10 15:11:52.045387 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.045275 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 10 15:11:52.045387 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.045312 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 10 15:11:52.045639 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.045622 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 10 15:11:52.045639 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.045637 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dpwr2\"" Jun 10 15:11:52.053668 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.053641 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx"] Jun 10 15:11:52.058789 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.058753 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:52.186066 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.186019 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.186253 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.186112 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spcp4\" (UniqueName: \"kubernetes.io/projected/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-kube-api-access-spcp4\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.186253 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.186229 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.186328 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.186262 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.186328 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.186285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.186396 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.186350 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.186396 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.186378 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.287392 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.287305 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.287392 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.287370 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.287614 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.287402 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.287614 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.287433 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.287614 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.287467 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.287614 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.287599 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.287821 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.287676 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spcp4\" (UniqueName: \"kubernetes.io/projected/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-kube-api-access-spcp4\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.288163 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.288137 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.290262 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.290237 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.290375 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.290323 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.290664 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.290441 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.290664 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.290438 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.297202 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.297183 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.297202 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.297198 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spcp4\" (UniqueName: \"kubernetes.io/projected/c216b24b-bef9-48d8-8c53-3bfe53ba5e7e-kube-api-access-spcp4\") pod \"istiod-openshift-gateway-75c67f8887-rpbbx\" (UID: \"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.350376 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.350349 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:52.495048 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.495015 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx"] Jun 10 15:11:52.496762 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:11:52.496727 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc216b24b_bef9_48d8_8c53_3bfe53ba5e7e.slice/crio-8881b915a9edc95be946572cc595dd9332e9373ac396fd9b96fb10f4f57432b3 WatchSource:0}: Error finding container 8881b915a9edc95be946572cc595dd9332e9373ac396fd9b96fb10f4f57432b3: Status 404 returned error can't find the container with id 8881b915a9edc95be946572cc595dd9332e9373ac396fd9b96fb10f4f57432b3 Jun 10 15:11:52.800796 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:52.800758 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" event={"ID":"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e","Type":"ContainerStarted","Data":"8881b915a9edc95be946572cc595dd9332e9373ac396fd9b96fb10f4f57432b3"} Jun 10 15:11:53.052431 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:53.052341 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:54.644352 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:54.644320 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zsddd" Jun 10 15:11:54.647816 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:54.647796 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-5z7pm" Jun 10 15:11:55.446108 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:55.446066 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 10 15:11:55.446196 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:55.446136 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 10 15:11:55.819185 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:55.819153 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" event={"ID":"c216b24b-bef9-48d8-8c53-3bfe53ba5e7e","Type":"ContainerStarted","Data":"1c276757cb39127d34c0c0e75f28908bb19735aa6c81f547c4d591149881a52c"} Jun 10 15:11:55.819564 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:55.819217 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:11:55.845245 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:55.845189 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" podStartSLOduration=0.898254835 podStartE2EDuration="3.845169346s" totalImagesPullingTime="2.946914511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:11:55.842240622 +0000 UTC m=+618.974533422" watchObservedRunningTime="2026-06-10 15:11:55.845169346 +0000 UTC m=+618.977462142" Jun 10 15:11:56.552252 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:56.552201 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:56.650894 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:56.650855 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:56.771190 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:56.771159 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-mvqsk" Jun 10 15:11:56.829118 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:56.829027 2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-rpbbx container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 10 15:11:56.829118 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:56.829081 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" podUID="c216b24b-bef9-48d8-8c53-3bfe53ba5e7e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 10 15:11:56.907160 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:56.907128 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:57.161176 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:57.161091 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:57.270527 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:57.270493 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:11:59.828227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:11:59.828203 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-rpbbx" Jun 10 15:12:01.248318 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.248288 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-6f5db44995-7w9cb"] Jun 10 15:12:01.257339 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.257315 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" Jun 10 15:12:01.260125 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.260104 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hq7v4\"" Jun 10 15:12:01.260245 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.260122 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 10 15:12:01.263991 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.263964 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6f5db44995-7w9cb"] Jun 10 15:12:01.267465 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.267419 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:01.370513 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.370483 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmpsc\" (UniqueName: \"kubernetes.io/projected/65d70a14-227e-4142-837b-17fac5cfa766-kube-api-access-lmpsc\") pod \"dashboard-redirect-6f5db44995-7w9cb\" (UID: \"65d70a14-227e-4142-837b-17fac5cfa766\") " pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" Jun 10 15:12:01.370682 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.370524 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/65d70a14-227e-4142-837b-17fac5cfa766-redirect-config\") pod \"dashboard-redirect-6f5db44995-7w9cb\" (UID: \"65d70a14-227e-4142-837b-17fac5cfa766\") " pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" Jun 10 15:12:01.471459 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.471422 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmpsc\" (UniqueName: \"kubernetes.io/projected/65d70a14-227e-4142-837b-17fac5cfa766-kube-api-access-lmpsc\") pod \"dashboard-redirect-6f5db44995-7w9cb\" (UID: \"65d70a14-227e-4142-837b-17fac5cfa766\") " pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" Jun 10 15:12:01.471459 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.471464 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/65d70a14-227e-4142-837b-17fac5cfa766-redirect-config\") pod \"dashboard-redirect-6f5db44995-7w9cb\" (UID: \"65d70a14-227e-4142-837b-17fac5cfa766\") " pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" Jun 10 15:12:01.472165 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.472147 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/65d70a14-227e-4142-837b-17fac5cfa766-redirect-config\") pod \"dashboard-redirect-6f5db44995-7w9cb\" (UID: \"65d70a14-227e-4142-837b-17fac5cfa766\") " pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" Jun 10 15:12:01.482970 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.482923 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmpsc\" (UniqueName: \"kubernetes.io/projected/65d70a14-227e-4142-837b-17fac5cfa766-kube-api-access-lmpsc\") pod \"dashboard-redirect-6f5db44995-7w9cb\" (UID: \"65d70a14-227e-4142-837b-17fac5cfa766\") " pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" Jun 10 15:12:01.570752 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.570722 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" Jun 10 15:12:01.705403 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.705380 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6f5db44995-7w9cb"] Jun 10 15:12:01.707479 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:12:01.707452 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d70a14_227e_4142_837b_17fac5cfa766.slice/crio-c502c1f6f0af1a439efe56153637f6e53961bad1b83d6e4d44edd0a6f4c8a517 WatchSource:0}: Error finding container c502c1f6f0af1a439efe56153637f6e53961bad1b83d6e4d44edd0a6f4c8a517: Status 404 returned error can't find the container with id c502c1f6f0af1a439efe56153637f6e53961bad1b83d6e4d44edd0a6f4c8a517 Jun 10 15:12:01.855066 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:01.854986 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" event={"ID":"65d70a14-227e-4142-837b-17fac5cfa766","Type":"ContainerStarted","Data":"c502c1f6f0af1a439efe56153637f6e53961bad1b83d6e4d44edd0a6f4c8a517"} Jun 10 15:12:06.342170 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:06.342134 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:06.441580 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:06.441545 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:06.572105 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:06.572074 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:07.893244 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:07.893213 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" event={"ID":"65d70a14-227e-4142-837b-17fac5cfa766","Type":"ContainerStarted","Data":"b8e9f783125a31c3d1b7960d75ab961da743322b2f075df4581b071bf6454a5f"} Jun 10 15:12:07.911374 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:07.911330 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-6f5db44995-7w9cb" podStartSLOduration=0.872311508 podStartE2EDuration="6.911314269s" totalImagesPullingTime="6.039002761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:12:07.910410172 +0000 UTC m=+631.042702972" watchObservedRunningTime="2026-06-10 15:12:07.911314269 +0000 UTC m=+631.043607063" Jun 10 15:12:21.722979 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.722944 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w"] Jun 10 15:12:21.725826 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.725810 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:21.728238 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.728217 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 10 15:12:21.728343 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.728286 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 10 15:12:21.729101 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.729085 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9wncc\"" Jun 10 15:12:21.736730 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.736713 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w"] Jun 10 15:12:21.836264 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.836229 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28n48\" (UniqueName: \"kubernetes.io/projected/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-kube-api-access-28n48\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:21.836423 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.836272 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:21.836423 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.836367 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:21.937319 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.937286 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:21.937489 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.937363 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28n48\" (UniqueName: \"kubernetes.io/projected/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-kube-api-access-28n48\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:21.937489 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.937391 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:21.937742 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.937718 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:21.937816 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.937725 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:21.947430 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:21.947411 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28n48\" (UniqueName: \"kubernetes.io/projected/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-kube-api-access-28n48\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:22.035617 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.035547 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:22.180687 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.180630 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w"] Jun 10 15:12:22.183019 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:12:22.182991 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a5b9b4_5212_47da_a193_52d4cfcf8aa6.slice/crio-a41273661b636e335ad6f92012712ce7359ba120bd51696775978d51d316d644 WatchSource:0}: Error finding container a41273661b636e335ad6f92012712ce7359ba120bd51696775978d51d316d644: Status 404 returned error can't find the container with id a41273661b636e335ad6f92012712ce7359ba120bd51696775978d51d316d644 Jun 10 15:12:22.319837 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.319807 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6"] Jun 10 15:12:22.322466 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.322447 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.331625 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.331593 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6"] Jun 10 15:12:22.441045 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.441015 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.441201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.441130 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm7nn\" (UniqueName: \"kubernetes.io/projected/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-kube-api-access-qm7nn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.441201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.441177 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.541818 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.541784 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm7nn\" (UniqueName: \"kubernetes.io/projected/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-kube-api-access-qm7nn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.541975 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.541827 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.541975 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.541857 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.542291 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.542271 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.542336 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.542291 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.551176 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.551156 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm7nn\" (UniqueName: \"kubernetes.io/projected/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-kube-api-access-qm7nn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.652473 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.652378 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:22.786071 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.785977 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6"] Jun 10 15:12:22.788368 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:12:22.788326 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc9c3372_90f7_4ebf_b9e6_27112eb7350a.slice/crio-e0e6f90c511ecb5ee292f125feacafe746bc7bac46d07474b7d0b8a0bd8f456d WatchSource:0}: Error finding container e0e6f90c511ecb5ee292f125feacafe746bc7bac46d07474b7d0b8a0bd8f456d: Status 404 returned error can't find the container with id e0e6f90c511ecb5ee292f125feacafe746bc7bac46d07474b7d0b8a0bd8f456d Jun 10 15:12:22.917701 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.917639 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp"] Jun 10 15:12:22.920531 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.920516 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:22.930589 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.930566 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp"] Jun 10 15:12:22.945274 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.945243 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbmqr\" (UniqueName: \"kubernetes.io/projected/291116fb-12cf-4feb-b3aa-7f62565a3084-kube-api-access-wbmqr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:22.945387 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.945284 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:22.945467 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.945418 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:22.973630 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.973599 2561 generic.go:356] "Generic (PLEG): container finished" podID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerID="20d6b73829229d60143a43da9940e037526489d2c8251eaad2086233900c5280" exitCode=0 Jun 10 15:12:22.973766 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.973690 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" event={"ID":"bc9c3372-90f7-4ebf-b9e6-27112eb7350a","Type":"ContainerDied","Data":"20d6b73829229d60143a43da9940e037526489d2c8251eaad2086233900c5280"} Jun 10 15:12:22.973766 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.973752 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" event={"ID":"bc9c3372-90f7-4ebf-b9e6-27112eb7350a","Type":"ContainerStarted","Data":"e0e6f90c511ecb5ee292f125feacafe746bc7bac46d07474b7d0b8a0bd8f456d"} Jun 10 15:12:22.975593 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.975570 2561 generic.go:356] "Generic (PLEG): container finished" podID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerID="bf5568702c894f37630d156f06efefa110b427dffc1bf6f3c59fe85dba4e227f" exitCode=0 Jun 10 15:12:22.975709 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.975614 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" event={"ID":"75a5b9b4-5212-47da-a193-52d4cfcf8aa6","Type":"ContainerDied","Data":"bf5568702c894f37630d156f06efefa110b427dffc1bf6f3c59fe85dba4e227f"} Jun 10 15:12:22.975709 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:22.975656 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" event={"ID":"75a5b9b4-5212-47da-a193-52d4cfcf8aa6","Type":"ContainerStarted","Data":"a41273661b636e335ad6f92012712ce7359ba120bd51696775978d51d316d644"} Jun 10 15:12:23.046219 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.046170 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:23.046409 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.046251 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbmqr\" (UniqueName: \"kubernetes.io/projected/291116fb-12cf-4feb-b3aa-7f62565a3084-kube-api-access-wbmqr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:23.046409 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.046289 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:23.046576 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.046555 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:23.046637 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.046618 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:23.056348 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.056316 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbmqr\" (UniqueName: \"kubernetes.io/projected/291116fb-12cf-4feb-b3aa-7f62565a3084-kube-api-access-wbmqr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:23.276086 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.276004 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:23.317296 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.317262 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf"] Jun 10 15:12:23.326072 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.326040 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.334334 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.334196 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf"] Jun 10 15:12:23.349428 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.349400 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.349730 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.349704 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.349826 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.349765 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9889\" (UniqueName: \"kubernetes.io/projected/a4248e78-61d4-44a2-945d-6b15352df733-kube-api-access-t9889\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.426991 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.426964 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp"] Jun 10 15:12:23.428721 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:12:23.428680 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291116fb_12cf_4feb_b3aa_7f62565a3084.slice/crio-4d7d89ac33885671904dd2e8ce5a1c581a8b17cdb817aee8de7e24db37238dbc WatchSource:0}: Error finding container 4d7d89ac33885671904dd2e8ce5a1c581a8b17cdb817aee8de7e24db37238dbc: Status 404 returned error can't find the container with id 4d7d89ac33885671904dd2e8ce5a1c581a8b17cdb817aee8de7e24db37238dbc Jun 10 15:12:23.451021 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.450998 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.451127 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.451051 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.451127 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.451111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9889\" (UniqueName: \"kubernetes.io/projected/a4248e78-61d4-44a2-945d-6b15352df733-kube-api-access-t9889\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.451410 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.451386 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.451474 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.451414 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.461284 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.461258 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9889\" (UniqueName: \"kubernetes.io/projected/a4248e78-61d4-44a2-945d-6b15352df733-kube-api-access-t9889\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.640393 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.640360 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:23.900201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.900174 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf"] Jun 10 15:12:23.954774 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:12:23.954744 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4248e78_61d4_44a2_945d_6b15352df733.slice/crio-43df8818fda4132a4e7c798549e00895f48f0040933abde3a16f90f2ae49bd88 WatchSource:0}: Error finding container 43df8818fda4132a4e7c798549e00895f48f0040933abde3a16f90f2ae49bd88: Status 404 returned error can't find the container with id 43df8818fda4132a4e7c798549e00895f48f0040933abde3a16f90f2ae49bd88 Jun 10 15:12:23.982206 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.982180 2561 generic.go:356] "Generic (PLEG): container finished" podID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerID="8f360706f87219728b8cd9588b826a40602de42a5420cce1b638aec79f74b32f" exitCode=0 Jun 10 15:12:23.982317 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.982265 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" event={"ID":"291116fb-12cf-4feb-b3aa-7f62565a3084","Type":"ContainerDied","Data":"8f360706f87219728b8cd9588b826a40602de42a5420cce1b638aec79f74b32f"} Jun 10 15:12:23.982388 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.982326 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" event={"ID":"291116fb-12cf-4feb-b3aa-7f62565a3084","Type":"ContainerStarted","Data":"4d7d89ac33885671904dd2e8ce5a1c581a8b17cdb817aee8de7e24db37238dbc"} Jun 10 15:12:23.984003 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.983983 2561 generic.go:356] "Generic (PLEG): container finished" podID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerID="de13af1350b0215830dac5e23522eb866094c4f94ed5b00fa9c84b8b97f9663c" exitCode=0 Jun 10 15:12:23.984109 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.984068 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" event={"ID":"bc9c3372-90f7-4ebf-b9e6-27112eb7350a","Type":"ContainerDied","Data":"de13af1350b0215830dac5e23522eb866094c4f94ed5b00fa9c84b8b97f9663c"} Jun 10 15:12:23.985977 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.985954 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" event={"ID":"75a5b9b4-5212-47da-a193-52d4cfcf8aa6","Type":"ContainerDied","Data":"57ac68c2d899a899c699aeebce979606682e3c2e62ff9ec0f2350ffdff017b95"} Jun 10 15:12:23.986088 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.986015 2561 generic.go:356] "Generic (PLEG): container finished" podID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerID="57ac68c2d899a899c699aeebce979606682e3c2e62ff9ec0f2350ffdff017b95" exitCode=0 Jun 10 15:12:23.987453 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:23.987372 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" event={"ID":"a4248e78-61d4-44a2-945d-6b15352df733","Type":"ContainerStarted","Data":"43df8818fda4132a4e7c798549e00895f48f0040933abde3a16f90f2ae49bd88"} Jun 10 15:12:24.995048 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:24.995014 2561 generic.go:356] "Generic (PLEG): container finished" podID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerID="15e8f63edba9d8e84424646db6dfe70df900542624ee2d4f94cb1aa22869b82d" exitCode=0 Jun 10 15:12:24.995479 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:24.995099 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" event={"ID":"75a5b9b4-5212-47da-a193-52d4cfcf8aa6","Type":"ContainerDied","Data":"15e8f63edba9d8e84424646db6dfe70df900542624ee2d4f94cb1aa22869b82d"} Jun 10 15:12:24.996604 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:24.996581 2561 generic.go:356] "Generic (PLEG): container finished" podID="a4248e78-61d4-44a2-945d-6b15352df733" containerID="b52aba5804afa1c02ddd3a3b71924ae82557127ca33e26348c5ea09e5c9145ba" exitCode=0 Jun 10 15:12:24.996728 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:24.996666 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" event={"ID":"a4248e78-61d4-44a2-945d-6b15352df733","Type":"ContainerDied","Data":"b52aba5804afa1c02ddd3a3b71924ae82557127ca33e26348c5ea09e5c9145ba"} Jun 10 15:12:24.998850 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:24.998741 2561 generic.go:356] "Generic (PLEG): container finished" podID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerID="5369cbebcd24ae9169486a788486dfa7d1ec0c1dc50a0ad7406cf1a377ebe499" exitCode=0 Jun 10 15:12:24.998920 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:24.998838 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" event={"ID":"291116fb-12cf-4feb-b3aa-7f62565a3084","Type":"ContainerDied","Data":"5369cbebcd24ae9169486a788486dfa7d1ec0c1dc50a0ad7406cf1a377ebe499"} Jun 10 15:12:25.001254 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:25.001232 2561 generic.go:356] "Generic (PLEG): container finished" podID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerID="c1b2878633a0e12fd762640f7e5b5a6c6340b9cf4e7eb48784f76775f7f17c71" exitCode=0 Jun 10 15:12:25.001341 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:25.001264 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" event={"ID":"bc9c3372-90f7-4ebf-b9e6-27112eb7350a","Type":"ContainerDied","Data":"c1b2878633a0e12fd762640f7e5b5a6c6340b9cf4e7eb48784f76775f7f17c71"} Jun 10 15:12:26.009448 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.009359 2561 generic.go:356] "Generic (PLEG): container finished" podID="a4248e78-61d4-44a2-945d-6b15352df733" containerID="061b7e16903d389859c9d02c394437ee0d542041b9fe61a8ed63385e3cd6fea8" exitCode=0 Jun 10 15:12:26.009876 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.009443 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" event={"ID":"a4248e78-61d4-44a2-945d-6b15352df733","Type":"ContainerDied","Data":"061b7e16903d389859c9d02c394437ee0d542041b9fe61a8ed63385e3cd6fea8"} Jun 10 15:12:26.011744 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.011719 2561 generic.go:356] "Generic (PLEG): container finished" podID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerID="8a7dc5f41a92ce6da0061e4b572a0c68f7ff08e71bc9afe0d9ffb8ac438c1eb7" exitCode=0 Jun 10 15:12:26.011834 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.011798 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" event={"ID":"291116fb-12cf-4feb-b3aa-7f62565a3084","Type":"ContainerDied","Data":"8a7dc5f41a92ce6da0061e4b572a0c68f7ff08e71bc9afe0d9ffb8ac438c1eb7"} Jun 10 15:12:26.157316 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.157289 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:26.235441 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.235419 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:26.276882 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.276819 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-util\") pod \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " Jun 10 15:12:26.276882 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.276871 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-bundle\") pod \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " Jun 10 15:12:26.277082 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.276906 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28n48\" (UniqueName: \"kubernetes.io/projected/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-kube-api-access-28n48\") pod \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\" (UID: \"75a5b9b4-5212-47da-a193-52d4cfcf8aa6\") " Jun 10 15:12:26.277082 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.276920 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-util\") pod \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " Jun 10 15:12:26.277082 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.276958 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm7nn\" (UniqueName: \"kubernetes.io/projected/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-kube-api-access-qm7nn\") pod \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " Jun 10 15:12:26.277082 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.276983 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-bundle\") pod \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\" (UID: \"bc9c3372-90f7-4ebf-b9e6-27112eb7350a\") " Jun 10 15:12:26.277608 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.277557 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-bundle" (OuterVolumeSpecName: "bundle") pod "75a5b9b4-5212-47da-a193-52d4cfcf8aa6" (UID: "75a5b9b4-5212-47da-a193-52d4cfcf8aa6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:12:26.277739 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.277616 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-bundle" (OuterVolumeSpecName: "bundle") pod "bc9c3372-90f7-4ebf-b9e6-27112eb7350a" (UID: "bc9c3372-90f7-4ebf-b9e6-27112eb7350a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:12:26.279630 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.279607 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-kube-api-access-qm7nn" (OuterVolumeSpecName: "kube-api-access-qm7nn") pod "bc9c3372-90f7-4ebf-b9e6-27112eb7350a" (UID: "bc9c3372-90f7-4ebf-b9e6-27112eb7350a"). InnerVolumeSpecName "kube-api-access-qm7nn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:12:26.279711 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.279645 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-kube-api-access-28n48" (OuterVolumeSpecName: "kube-api-access-28n48") pod "75a5b9b4-5212-47da-a193-52d4cfcf8aa6" (UID: "75a5b9b4-5212-47da-a193-52d4cfcf8aa6"). InnerVolumeSpecName "kube-api-access-28n48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:12:26.283046 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.283027 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-util" (OuterVolumeSpecName: "util") pod "75a5b9b4-5212-47da-a193-52d4cfcf8aa6" (UID: "75a5b9b4-5212-47da-a193-52d4cfcf8aa6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:12:26.283225 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.283208 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-util" (OuterVolumeSpecName: "util") pod "bc9c3372-90f7-4ebf-b9e6-27112eb7350a" (UID: "bc9c3372-90f7-4ebf-b9e6-27112eb7350a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:12:26.377750 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.377702 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:26.377750 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.377746 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:26.377750 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.377757 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28n48\" (UniqueName: \"kubernetes.io/projected/75a5b9b4-5212-47da-a193-52d4cfcf8aa6-kube-api-access-28n48\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:26.378037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.377766 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:26.378037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.377775 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm7nn\" (UniqueName: \"kubernetes.io/projected/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-kube-api-access-qm7nn\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:26.378037 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:26.377784 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc9c3372-90f7-4ebf-b9e6-27112eb7350a-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:27.019500 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.019472 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" Jun 10 15:12:27.019948 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.019473 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6" event={"ID":"bc9c3372-90f7-4ebf-b9e6-27112eb7350a","Type":"ContainerDied","Data":"e0e6f90c511ecb5ee292f125feacafe746bc7bac46d07474b7d0b8a0bd8f456d"} Jun 10 15:12:27.019948 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.019578 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0e6f90c511ecb5ee292f125feacafe746bc7bac46d07474b7d0b8a0bd8f456d" Jun 10 15:12:27.021421 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.021404 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" Jun 10 15:12:27.021547 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.021418 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w" event={"ID":"75a5b9b4-5212-47da-a193-52d4cfcf8aa6","Type":"ContainerDied","Data":"a41273661b636e335ad6f92012712ce7359ba120bd51696775978d51d316d644"} Jun 10 15:12:27.021547 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.021451 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a41273661b636e335ad6f92012712ce7359ba120bd51696775978d51d316d644" Jun 10 15:12:27.023893 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.023874 2561 generic.go:356] "Generic (PLEG): container finished" podID="a4248e78-61d4-44a2-945d-6b15352df733" containerID="fcc3c888a68b1256da9a20b8c5b8df10da2244aaf317c07c9f974dc0c6d96ff1" exitCode=0 Jun 10 15:12:27.024003 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.023966 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" event={"ID":"a4248e78-61d4-44a2-945d-6b15352df733","Type":"ContainerDied","Data":"fcc3c888a68b1256da9a20b8c5b8df10da2244aaf317c07c9f974dc0c6d96ff1"} Jun 10 15:12:27.155429 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.155409 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:27.184913 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.184882 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-util\") pod \"291116fb-12cf-4feb-b3aa-7f62565a3084\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " Jun 10 15:12:27.185061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.184923 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbmqr\" (UniqueName: \"kubernetes.io/projected/291116fb-12cf-4feb-b3aa-7f62565a3084-kube-api-access-wbmqr\") pod \"291116fb-12cf-4feb-b3aa-7f62565a3084\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " Jun 10 15:12:27.185061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.185003 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-bundle\") pod \"291116fb-12cf-4feb-b3aa-7f62565a3084\" (UID: \"291116fb-12cf-4feb-b3aa-7f62565a3084\") " Jun 10 15:12:27.185782 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.185751 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-bundle" (OuterVolumeSpecName: "bundle") pod "291116fb-12cf-4feb-b3aa-7f62565a3084" (UID: "291116fb-12cf-4feb-b3aa-7f62565a3084"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:12:27.187124 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.187090 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/291116fb-12cf-4feb-b3aa-7f62565a3084-kube-api-access-wbmqr" (OuterVolumeSpecName: "kube-api-access-wbmqr") pod "291116fb-12cf-4feb-b3aa-7f62565a3084" (UID: "291116fb-12cf-4feb-b3aa-7f62565a3084"). InnerVolumeSpecName "kube-api-access-wbmqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:12:27.190404 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.190379 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-util" (OuterVolumeSpecName: "util") pod "291116fb-12cf-4feb-b3aa-7f62565a3084" (UID: "291116fb-12cf-4feb-b3aa-7f62565a3084"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:12:27.286669 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.286567 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:27.286669 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.286598 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbmqr\" (UniqueName: \"kubernetes.io/projected/291116fb-12cf-4feb-b3aa-7f62565a3084-kube-api-access-wbmqr\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:27.286669 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:27.286610 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/291116fb-12cf-4feb-b3aa-7f62565a3084-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:28.032777 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.032743 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" Jun 10 15:12:28.032777 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.032743 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp" event={"ID":"291116fb-12cf-4feb-b3aa-7f62565a3084","Type":"ContainerDied","Data":"4d7d89ac33885671904dd2e8ce5a1c581a8b17cdb817aee8de7e24db37238dbc"} Jun 10 15:12:28.033281 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.032809 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d7d89ac33885671904dd2e8ce5a1c581a8b17cdb817aee8de7e24db37238dbc" Jun 10 15:12:28.165393 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.165364 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:28.296917 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.296892 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-util\") pod \"a4248e78-61d4-44a2-945d-6b15352df733\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " Jun 10 15:12:28.297085 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.296978 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-bundle\") pod \"a4248e78-61d4-44a2-945d-6b15352df733\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " Jun 10 15:12:28.297085 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.297002 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9889\" (UniqueName: \"kubernetes.io/projected/a4248e78-61d4-44a2-945d-6b15352df733-kube-api-access-t9889\") pod \"a4248e78-61d4-44a2-945d-6b15352df733\" (UID: \"a4248e78-61d4-44a2-945d-6b15352df733\") " Jun 10 15:12:28.297521 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.297500 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-bundle" (OuterVolumeSpecName: "bundle") pod "a4248e78-61d4-44a2-945d-6b15352df733" (UID: "a4248e78-61d4-44a2-945d-6b15352df733"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:12:28.299172 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.299139 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4248e78-61d4-44a2-945d-6b15352df733-kube-api-access-t9889" (OuterVolumeSpecName: "kube-api-access-t9889") pod "a4248e78-61d4-44a2-945d-6b15352df733" (UID: "a4248e78-61d4-44a2-945d-6b15352df733"). InnerVolumeSpecName "kube-api-access-t9889". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:12:28.304720 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.304699 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-util" (OuterVolumeSpecName: "util") pod "a4248e78-61d4-44a2-945d-6b15352df733" (UID: "a4248e78-61d4-44a2-945d-6b15352df733"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:12:28.397746 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.397720 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9889\" (UniqueName: \"kubernetes.io/projected/a4248e78-61d4-44a2-945d-6b15352df733-kube-api-access-t9889\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:28.397746 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.397743 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:28.397880 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:28.397757 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4248e78-61d4-44a2-945d-6b15352df733-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:12:29.045104 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:29.045074 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" Jun 10 15:12:29.045104 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:29.045088 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf" event={"ID":"a4248e78-61d4-44a2-945d-6b15352df733","Type":"ContainerDied","Data":"43df8818fda4132a4e7c798549e00895f48f0040933abde3a16f90f2ae49bd88"} Jun 10 15:12:29.045557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:29.045172 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43df8818fda4132a4e7c798549e00895f48f0040933abde3a16f90f2ae49bd88" Jun 10 15:12:34.741671 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:34.741646 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:35.337111 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:35.337081 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:36.410814 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:36.410774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:36.558060 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:36.558018 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:39.341293 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:39.341256 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:42.570494 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570459 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq"] Jun 10 15:12:42.570904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570839 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4248e78-61d4-44a2-945d-6b15352df733" containerName="pull" Jun 10 15:12:42.570904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570852 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4248e78-61d4-44a2-945d-6b15352df733" containerName="pull" Jun 10 15:12:42.570904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570869 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerName="pull" Jun 10 15:12:42.570904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570875 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerName="pull" Jun 10 15:12:42.570904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570885 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerName="util" Jun 10 15:12:42.570904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570891 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerName="util" Jun 10 15:12:42.570904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570897 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerName="extract" Jun 10 15:12:42.570904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570902 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerName="extract" Jun 10 15:12:42.570904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570908 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerName="extract" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570913 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerName="extract" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570922 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerName="util" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570959 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerName="util" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570966 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerName="pull" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570972 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerName="pull" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570985 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4248e78-61d4-44a2-945d-6b15352df733" containerName="util" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570990 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4248e78-61d4-44a2-945d-6b15352df733" containerName="util" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.570997 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerName="pull" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571002 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerName="pull" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571009 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerName="extract" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571014 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerName="extract" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571023 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerName="util" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571027 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerName="util" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571033 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4248e78-61d4-44a2-945d-6b15352df733" containerName="extract" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571038 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4248e78-61d4-44a2-945d-6b15352df733" containerName="extract" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571105 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc9c3372-90f7-4ebf-b9e6-27112eb7350a" containerName="extract" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571113 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="291116fb-12cf-4feb-b3aa-7f62565a3084" containerName="extract" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571120 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4248e78-61d4-44a2-945d-6b15352df733" containerName="extract" Jun 10 15:12:42.571234 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.571127 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="75a5b9b4-5212-47da-a193-52d4cfcf8aa6" containerName="extract" Jun 10 15:12:42.590441 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.590416 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:12:42.593359 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.593277 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9zgmq\"" Jun 10 15:12:42.593824 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.593802 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 10 15:12:42.594018 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.593830 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 10 15:12:42.611288 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.611262 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq"] Jun 10 15:12:42.644322 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.644295 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:42.708349 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.708323 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" (UID: \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:12:42.708477 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.708410 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgvxd\" (UniqueName: \"kubernetes.io/projected/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-kube-api-access-rgvxd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" (UID: \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:12:42.809026 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.809002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" (UID: \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:12:42.809127 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.809053 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgvxd\" (UniqueName: \"kubernetes.io/projected/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-kube-api-access-rgvxd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" (UID: \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:12:42.809400 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.809379 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" (UID: \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:12:42.823627 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.823562 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgvxd\" (UniqueName: \"kubernetes.io/projected/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-kube-api-access-rgvxd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" (UID: \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:12:42.901510 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:42.901486 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:12:43.039850 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:43.039826 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq"] Jun 10 15:12:43.041617 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:12:43.041590 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce29c0a9_f9a3_4775_8bdb_31a1ef54d8c4.slice/crio-c0a6694b299b9dccf5fcdfb6959047a9fba918c12f041e324206796b5042a290 WatchSource:0}: Error finding container c0a6694b299b9dccf5fcdfb6959047a9fba918c12f041e324206796b5042a290: Status 404 returned error can't find the container with id c0a6694b299b9dccf5fcdfb6959047a9fba918c12f041e324206796b5042a290 Jun 10 15:12:43.126759 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:43.126698 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" event={"ID":"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4","Type":"ContainerStarted","Data":"c0a6694b299b9dccf5fcdfb6959047a9fba918c12f041e324206796b5042a290"} Jun 10 15:12:43.348365 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:43.348331 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:45.364903 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:45.364866 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:45.591523 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:45.591488 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:47.989409 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:47.989373 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:49.161878 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:49.161844 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:49.266186 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:49.266153 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:49.553292 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:49.553265 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:49.631009 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:49.630980 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:53.190777 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:53.190732 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" event={"ID":"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4","Type":"ContainerStarted","Data":"ffb86ea69dd2d154773d0133bd7067de657cf0318c2991f096610a764936f680"} Jun 10 15:12:53.191150 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:53.190812 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:12:53.222623 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:53.222575 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" podStartSLOduration=2.11686147 podStartE2EDuration="11.222561928s" totalImagesPullingTime="9.105700458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:12:53.220000427 +0000 UTC m=+676.352293417" watchObservedRunningTime="2026-06-10 15:12:53.222561928 +0000 UTC m=+676.354854721" Jun 10 15:12:53.463388 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:53.463303 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:53.599526 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:53.599497 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:55.408196 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:55.408148 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:12:56.365594 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:12:56.365557 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:00.321754 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:00.321724 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:00.448901 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:00.448871 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:02.731730 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:02.731693 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:04.198818 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:04.198784 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:05.035914 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:05.035880 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:05.139074 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:05.139047 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:06.007979 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.007943 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq"] Jun 10 15:13:06.008402 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.008157 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" containerName="manager" containerID="cri-o://ffb86ea69dd2d154773d0133bd7067de657cf0318c2991f096610a764936f680" gracePeriod=2 Jun 10 15:13:06.012330 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.012300 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq"] Jun 10 15:13:06.013840 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:13:06.013800 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" is forbidden: User \"system:node:ip-10-0-129-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-85.ec2.internal' and this object" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:06.069483 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.069450 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv"] Jun 10 15:13:06.070075 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.070054 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" containerName="manager" Jun 10 15:13:06.070130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.070080 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" containerName="manager" Jun 10 15:13:06.070208 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.070195 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" containerName="manager" Jun 10 15:13:06.073063 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.073046 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:06.088743 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:13:06.088716 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" is forbidden: User \"system:node:ip-10-0-129-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-85.ec2.internal' and this object" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:06.112735 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.112709 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv"] Jun 10 15:13:06.192885 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.192858 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d6a398f-915c-4ad4-9cba-014c97750648-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4ljv\" (UID: \"2d6a398f-915c-4ad4-9cba-014c97750648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:06.193050 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.192901 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rthm\" (UniqueName: \"kubernetes.io/projected/2d6a398f-915c-4ad4-9cba-014c97750648-kube-api-access-8rthm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4ljv\" (UID: \"2d6a398f-915c-4ad4-9cba-014c97750648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:06.265597 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.265530 2561 generic.go:356] "Generic (PLEG): container finished" podID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" containerID="ffb86ea69dd2d154773d0133bd7067de657cf0318c2991f096610a764936f680" exitCode=0 Jun 10 15:13:06.293600 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.293579 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d6a398f-915c-4ad4-9cba-014c97750648-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4ljv\" (UID: \"2d6a398f-915c-4ad4-9cba-014c97750648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:06.293702 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.293616 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rthm\" (UniqueName: \"kubernetes.io/projected/2d6a398f-915c-4ad4-9cba-014c97750648-kube-api-access-8rthm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4ljv\" (UID: \"2d6a398f-915c-4ad4-9cba-014c97750648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:06.293963 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.293942 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d6a398f-915c-4ad4-9cba-014c97750648-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4ljv\" (UID: \"2d6a398f-915c-4ad4-9cba-014c97750648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:06.301496 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.301477 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:06.304096 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:13:06.304071 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" is forbidden: User \"system:node:ip-10-0-129-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-85.ec2.internal' and this object" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:06.305174 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.305154 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rthm\" (UniqueName: \"kubernetes.io/projected/2d6a398f-915c-4ad4-9cba-014c97750648-kube-api-access-8rthm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4ljv\" (UID: \"2d6a398f-915c-4ad4-9cba-014c97750648\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:06.394364 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.394343 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgvxd\" (UniqueName: \"kubernetes.io/projected/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-kube-api-access-rgvxd\") pod \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\" (UID: \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\") " Jun 10 15:13:06.394473 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.394424 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-extensions-socket-volume\") pod \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\" (UID: \"ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4\") " Jun 10 15:13:06.394892 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.394870 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" (UID: "ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:13:06.396229 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.396208 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-kube-api-access-rgvxd" (OuterVolumeSpecName: "kube-api-access-rgvxd") pod "ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" (UID: "ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4"). InnerVolumeSpecName "kube-api-access-rgvxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:13:06.409711 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.409689 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:06.496028 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.495774 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-extensions-socket-volume\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:13:06.496028 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.495806 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgvxd\" (UniqueName: \"kubernetes.io/projected/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4-kube-api-access-rgvxd\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:13:06.560213 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:06.560182 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv"] Jun 10 15:13:06.562687 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:13:06.562650 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6a398f_915c_4ad4_9cba_014c97750648.slice/crio-b58bdec6fd108f8f7c29baab80eca647ac287b96dd4dbf8c2c303814ef7bffde WatchSource:0}: Error finding container b58bdec6fd108f8f7c29baab80eca647ac287b96dd4dbf8c2c303814ef7bffde: Status 404 returned error can't find the container with id b58bdec6fd108f8f7c29baab80eca647ac287b96dd4dbf8c2c303814ef7bffde Jun 10 15:13:07.272565 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:07.272536 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:07.272565 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:07.272568 2561 scope.go:117] "RemoveContainer" containerID="ffb86ea69dd2d154773d0133bd7067de657cf0318c2991f096610a764936f680" Jun 10 15:13:07.274583 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:07.274560 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" event={"ID":"2d6a398f-915c-4ad4-9cba-014c97750648","Type":"ContainerStarted","Data":"8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c"} Jun 10 15:13:07.274688 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:07.274606 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" event={"ID":"2d6a398f-915c-4ad4-9cba-014c97750648","Type":"ContainerStarted","Data":"b58bdec6fd108f8f7c29baab80eca647ac287b96dd4dbf8c2c303814ef7bffde"} Jun 10 15:13:07.274737 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:07.274698 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:07.276105 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:13:07.276071 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" is forbidden: User \"system:node:ip-10-0-129-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-85.ec2.internal' and this object" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:07.305945 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:07.305866 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" podStartSLOduration=1.3058526289999999 podStartE2EDuration="1.305852629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:13:07.304798986 +0000 UTC m=+690.437091801" watchObservedRunningTime="2026-06-10 15:13:07.305852629 +0000 UTC m=+690.438145422" Jun 10 15:13:07.306879 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:13:07.306851 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" is forbidden: User \"system:node:ip-10-0-129-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-85.ec2.internal' and this object" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:07.308767 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:13:07.308744 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" is forbidden: User \"system:node:ip-10-0-129-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-85.ec2.internal' and this object" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:07.343497 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:07.343472 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" path="/var/lib/kubelet/pods/ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4/volumes" Jun 10 15:13:07.344135 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:13:07.344116 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t9hfq\" is forbidden: User \"system:node:ip-10-0-129-85.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-85.ec2.internal' and this object" podUID="ce29c0a9-f9a3-4775-8bdb-31a1ef54d8c4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t9hfq" Jun 10 15:13:08.849459 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:08.849424 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:09.056053 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:09.056020 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:09.175965 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:09.175871 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:09.281256 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:09.281232 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:11.435813 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:11.435778 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:11.725097 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:11.725021 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:11.822897 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:11.822866 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:18.223549 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:18.223467 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:18.284606 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:18.284581 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:18.314258 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:18.314228 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:22.958551 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:22.958502 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:23.055289 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:23.055250 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:26.764162 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:26.764131 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:26.852289 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:26.852259 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:27.181328 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:27.181292 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:27.288414 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:27.288380 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:31.174782 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:31.174741 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:31.273021 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:31.272994 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:34.924582 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:34.924549 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv"] Jun 10 15:13:34.925054 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:34.924734 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" podUID="2d6a398f-915c-4ad4-9cba-014c97750648" containerName="manager" containerID="cri-o://8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c" gracePeriod=10 Jun 10 15:13:35.176341 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.176289 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:35.211159 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.211135 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d6a398f-915c-4ad4-9cba-014c97750648-extensions-socket-volume\") pod \"2d6a398f-915c-4ad4-9cba-014c97750648\" (UID: \"2d6a398f-915c-4ad4-9cba-014c97750648\") " Jun 10 15:13:35.211274 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.211198 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rthm\" (UniqueName: \"kubernetes.io/projected/2d6a398f-915c-4ad4-9cba-014c97750648-kube-api-access-8rthm\") pod \"2d6a398f-915c-4ad4-9cba-014c97750648\" (UID: \"2d6a398f-915c-4ad4-9cba-014c97750648\") " Jun 10 15:13:35.211518 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.211494 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d6a398f-915c-4ad4-9cba-014c97750648-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2d6a398f-915c-4ad4-9cba-014c97750648" (UID: "2d6a398f-915c-4ad4-9cba-014c97750648"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:13:35.213075 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.213053 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d6a398f-915c-4ad4-9cba-014c97750648-kube-api-access-8rthm" (OuterVolumeSpecName: "kube-api-access-8rthm") pod "2d6a398f-915c-4ad4-9cba-014c97750648" (UID: "2d6a398f-915c-4ad4-9cba-014c97750648"). InnerVolumeSpecName "kube-api-access-8rthm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:13:35.311718 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.311693 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d6a398f-915c-4ad4-9cba-014c97750648-extensions-socket-volume\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:13:35.311718 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.311715 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rthm\" (UniqueName: \"kubernetes.io/projected/2d6a398f-915c-4ad4-9cba-014c97750648-kube-api-access-8rthm\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:13:35.398403 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.398373 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q"] Jun 10 15:13:35.398741 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.398728 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d6a398f-915c-4ad4-9cba-014c97750648" containerName="manager" Jun 10 15:13:35.398788 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.398743 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d6a398f-915c-4ad4-9cba-014c97750648" containerName="manager" Jun 10 15:13:35.398824 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.398818 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d6a398f-915c-4ad4-9cba-014c97750648" containerName="manager" Jun 10 15:13:35.400921 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.400905 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:35.418594 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.418570 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q"] Jun 10 15:13:35.439521 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.439463 2561 generic.go:356] "Generic (PLEG): container finished" podID="2d6a398f-915c-4ad4-9cba-014c97750648" containerID="8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c" exitCode=0 Jun 10 15:13:35.439521 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.439494 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" event={"ID":"2d6a398f-915c-4ad4-9cba-014c97750648","Type":"ContainerDied","Data":"8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c"} Jun 10 15:13:35.439521 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.439511 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" Jun 10 15:13:35.439521 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.439522 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv" event={"ID":"2d6a398f-915c-4ad4-9cba-014c97750648","Type":"ContainerDied","Data":"b58bdec6fd108f8f7c29baab80eca647ac287b96dd4dbf8c2c303814ef7bffde"} Jun 10 15:13:35.439741 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.439543 2561 scope.go:117] "RemoveContainer" containerID="8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c" Jun 10 15:13:35.448403 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.448388 2561 scope.go:117] "RemoveContainer" containerID="8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c" Jun 10 15:13:35.448741 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:13:35.448723 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c\": container with ID starting with 8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c not found: ID does not exist" containerID="8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c" Jun 10 15:13:35.448795 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.448749 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c"} err="failed to get container status \"8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c\": rpc error: code = NotFound desc = could not find container \"8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c\": container with ID starting with 8fbed74fb57e1ceb9e0d9aaeea28da0dd04c0f70d7201af88768080d0a24797c not found: ID does not exist" Jun 10 15:13:35.466869 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.466847 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv"] Jun 10 15:13:35.474277 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.474257 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4ljv"] Jun 10 15:13:35.514022 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.514003 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2q2z\" (UniqueName: \"kubernetes.io/projected/9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d-kube-api-access-g2q2z\") pod \"kuadrant-operator-controller-manager-86878c769f-hwf5q\" (UID: \"9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:35.514105 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.514058 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hwf5q\" (UID: \"9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:35.615272 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.615243 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hwf5q\" (UID: \"9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:35.615393 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.615316 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2q2z\" (UniqueName: \"kubernetes.io/projected/9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d-kube-api-access-g2q2z\") pod \"kuadrant-operator-controller-manager-86878c769f-hwf5q\" (UID: \"9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:35.615578 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.615558 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hwf5q\" (UID: \"9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:35.625514 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.625491 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2q2z\" (UniqueName: \"kubernetes.io/projected/9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d-kube-api-access-g2q2z\") pod \"kuadrant-operator-controller-manager-86878c769f-hwf5q\" (UID: \"9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:35.711617 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.711562 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:35.845493 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:35.845466 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q"] Jun 10 15:13:35.846568 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:13:35.846544 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c97b7e1_7ce3_4bc0_b932_ff4eadb5031d.slice/crio-c61fdbc7970a811d286efe904707bcc2ed21ce50f31945bab6e879f02250627e WatchSource:0}: Error finding container c61fdbc7970a811d286efe904707bcc2ed21ce50f31945bab6e879f02250627e: Status 404 returned error can't find the container with id c61fdbc7970a811d286efe904707bcc2ed21ce50f31945bab6e879f02250627e Jun 10 15:13:36.447145 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:36.447105 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" event={"ID":"9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d","Type":"ContainerStarted","Data":"15609e41534c563db289e380ed980ad426da2a5180add7a3967ba6c8e5ea7a07"} Jun 10 15:13:36.447500 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:36.447165 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" event={"ID":"9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d","Type":"ContainerStarted","Data":"c61fdbc7970a811d286efe904707bcc2ed21ce50f31945bab6e879f02250627e"} Jun 10 15:13:36.447500 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:36.447197 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:36.472909 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:36.472867 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" podStartSLOduration=1.472852725 podStartE2EDuration="1.472852725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:13:36.469407984 +0000 UTC m=+719.601700777" watchObservedRunningTime="2026-06-10 15:13:36.472852725 +0000 UTC m=+719.605145517" Jun 10 15:13:37.277444 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:37.277413 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:37.344734 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:37.344710 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d6a398f-915c-4ad4-9cba-014c97750648" path="/var/lib/kubelet/pods/2d6a398f-915c-4ad4-9cba-014c97750648/volumes" Jun 10 15:13:37.375209 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:37.375186 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:41.977330 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:41.977291 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:42.078057 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:42.078011 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:47.454518 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:47.454494 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwf5q" Jun 10 15:13:51.512649 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:51.512616 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:51.611858 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:51.611824 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:55.083688 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.083657 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:55.595381 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.595347 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:55.603605 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.603568 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dwdvm"] Jun 10 15:13:55.606118 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.606103 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:13:55.608727 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.608710 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 10 15:13:55.608820 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.608795 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9wncc\"" Jun 10 15:13:55.617447 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.617428 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dwdvm"] Jun 10 15:13:55.663246 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.663215 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94f0a909-e0b8-4e5b-9e5a-f1a650185327-config-file\") pod \"limitador-limitador-64c94fd64f-dwdvm\" (UID: \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:13:55.663404 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.663285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmpdw\" (UniqueName: \"kubernetes.io/projected/94f0a909-e0b8-4e5b-9e5a-f1a650185327-kube-api-access-mmpdw\") pod \"limitador-limitador-64c94fd64f-dwdvm\" (UID: \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:13:55.699238 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.699212 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dwdvm"] Jun 10 15:13:55.708175 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.708149 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:55.764135 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.764114 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmpdw\" (UniqueName: \"kubernetes.io/projected/94f0a909-e0b8-4e5b-9e5a-f1a650185327-kube-api-access-mmpdw\") pod \"limitador-limitador-64c94fd64f-dwdvm\" (UID: \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:13:55.764236 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.764195 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94f0a909-e0b8-4e5b-9e5a-f1a650185327-config-file\") pod \"limitador-limitador-64c94fd64f-dwdvm\" (UID: \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:13:55.764794 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.764776 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94f0a909-e0b8-4e5b-9e5a-f1a650185327-config-file\") pod \"limitador-limitador-64c94fd64f-dwdvm\" (UID: \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:13:55.773475 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.773455 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmpdw\" (UniqueName: \"kubernetes.io/projected/94f0a909-e0b8-4e5b-9e5a-f1a650185327-kube-api-access-mmpdw\") pod \"limitador-limitador-64c94fd64f-dwdvm\" (UID: \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:13:55.777201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.777180 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:55.916209 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:55.916155 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:13:56.050079 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:56.050055 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dwdvm"] Jun 10 15:13:56.052389 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:13:56.052360 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f0a909_e0b8_4e5b_9e5a_f1a650185327.slice/crio-4ca20bb3afc19228c0b63f71a65f3ee96aff1c6eeee06cc24f0168bfa0818a54 WatchSource:0}: Error finding container 4ca20bb3afc19228c0b63f71a65f3ee96aff1c6eeee06cc24f0168bfa0818a54: Status 404 returned error can't find the container with id 4ca20bb3afc19228c0b63f71a65f3ee96aff1c6eeee06cc24f0168bfa0818a54 Jun 10 15:13:56.054435 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:56.054419 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 15:13:56.512146 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:56.512113 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:56.554040 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:56.554008 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" event={"ID":"94f0a909-e0b8-4e5b-9e5a-f1a650185327","Type":"ContainerStarted","Data":"4ca20bb3afc19228c0b63f71a65f3ee96aff1c6eeee06cc24f0168bfa0818a54"} Jun 10 15:13:56.596239 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:56.596214 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:13:59.574117 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:59.574079 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" event={"ID":"94f0a909-e0b8-4e5b-9e5a-f1a650185327","Type":"ContainerStarted","Data":"0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e"} Jun 10 15:13:59.574473 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:59.574140 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:13:59.595242 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:13:59.595190 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" podStartSLOduration=1.937452368 podStartE2EDuration="4.595173282s" totalImagesPullingTime="2.657720914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:13:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:13:59.59434267 +0000 UTC m=+742.726635465" watchObservedRunningTime="2026-06-10 15:13:59.595173282 +0000 UTC m=+742.727466075" Jun 10 15:14:03.096990 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:03.096962 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:14:10.581945 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:10.581911 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:14:11.070781 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.070750 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dwdvm"] Jun 10 15:14:11.071028 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.070992 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" podUID="94f0a909-e0b8-4e5b-9e5a-f1a650185327" containerName="limitador" containerID="cri-o://0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e" gracePeriod=30 Jun 10 15:14:11.615187 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.615161 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:14:11.640903 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.640845 2561 generic.go:356] "Generic (PLEG): container finished" podID="94f0a909-e0b8-4e5b-9e5a-f1a650185327" containerID="0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e" exitCode=0 Jun 10 15:14:11.640903 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.640879 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" Jun 10 15:14:11.641061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.640944 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" event={"ID":"94f0a909-e0b8-4e5b-9e5a-f1a650185327","Type":"ContainerDied","Data":"0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e"} Jun 10 15:14:11.641061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.640995 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dwdvm" event={"ID":"94f0a909-e0b8-4e5b-9e5a-f1a650185327","Type":"ContainerDied","Data":"4ca20bb3afc19228c0b63f71a65f3ee96aff1c6eeee06cc24f0168bfa0818a54"} Jun 10 15:14:11.641061 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.641018 2561 scope.go:117] "RemoveContainer" containerID="0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e" Jun 10 15:14:11.649343 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.649328 2561 scope.go:117] "RemoveContainer" containerID="0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e" Jun 10 15:14:11.649586 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:14:11.649568 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e\": container with ID starting with 0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e not found: ID does not exist" containerID="0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e" Jun 10 15:14:11.649674 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.649591 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e"} err="failed to get container status \"0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e\": rpc error: code = NotFound desc = could not find container \"0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e\": container with ID starting with 0b3831979a92d03771a945f4e6613d8b36e9ff3f58efb3c889c0f96e80f1445e not found: ID does not exist" Jun 10 15:14:11.680619 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.680598 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmpdw\" (UniqueName: \"kubernetes.io/projected/94f0a909-e0b8-4e5b-9e5a-f1a650185327-kube-api-access-mmpdw\") pod \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\" (UID: \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\") " Jun 10 15:14:11.680721 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.680634 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94f0a909-e0b8-4e5b-9e5a-f1a650185327-config-file\") pod \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\" (UID: \"94f0a909-e0b8-4e5b-9e5a-f1a650185327\") " Jun 10 15:14:11.681001 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.680974 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94f0a909-e0b8-4e5b-9e5a-f1a650185327-config-file" (OuterVolumeSpecName: "config-file") pod "94f0a909-e0b8-4e5b-9e5a-f1a650185327" (UID: "94f0a909-e0b8-4e5b-9e5a-f1a650185327"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:14:11.682512 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.682494 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94f0a909-e0b8-4e5b-9e5a-f1a650185327-kube-api-access-mmpdw" (OuterVolumeSpecName: "kube-api-access-mmpdw") pod "94f0a909-e0b8-4e5b-9e5a-f1a650185327" (UID: "94f0a909-e0b8-4e5b-9e5a-f1a650185327"). InnerVolumeSpecName "kube-api-access-mmpdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:14:11.781602 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.781574 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmpdw\" (UniqueName: \"kubernetes.io/projected/94f0a909-e0b8-4e5b-9e5a-f1a650185327-kube-api-access-mmpdw\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:14:11.781602 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.781598 2561 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/94f0a909-e0b8-4e5b-9e5a-f1a650185327-config-file\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:14:11.965741 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.965714 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dwdvm"] Jun 10 15:14:11.970534 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:11.970512 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dwdvm"] Jun 10 15:14:13.343077 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:13.343043 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94f0a909-e0b8-4e5b-9e5a-f1a650185327" path="/var/lib/kubelet/pods/94f0a909-e0b8-4e5b-9e5a-f1a650185327/volumes" Jun 10 15:14:16.697463 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:16.697425 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:14:16.794796 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:16.794756 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:14:17.160100 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.160069 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92"] Jun 10 15:14:17.160466 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.160453 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94f0a909-e0b8-4e5b-9e5a-f1a650185327" containerName="limitador" Jun 10 15:14:17.160512 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.160470 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="94f0a909-e0b8-4e5b-9e5a-f1a650185327" containerName="limitador" Jun 10 15:14:17.160562 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.160550 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="94f0a909-e0b8-4e5b-9e5a-f1a650185327" containerName="limitador" Jun 10 15:14:17.163612 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.163595 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:17.166255 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.166228 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jun 10 15:14:17.166371 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.166305 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jun 10 15:14:17.166371 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.166316 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"default-dockercfg-4r7xp\"" Jun 10 15:14:17.166497 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.166315 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"postgres-creds\"" Jun 10 15:14:17.175415 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.175397 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92"] Jun 10 15:14:17.205589 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.205563 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:14:17.324438 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.324411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2mz8\" (UniqueName: \"kubernetes.io/projected/83f7a007-f5b5-4342-afac-4c971e6b413e-kube-api-access-x2mz8\") pod \"postgres-69fb4594cf-zlh92\" (UID: \"83f7a007-f5b5-4342-afac-4c971e6b413e\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:17.324546 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.324481 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/83f7a007-f5b5-4342-afac-4c971e6b413e-data\") pod \"postgres-69fb4594cf-zlh92\" (UID: \"83f7a007-f5b5-4342-afac-4c971e6b413e\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:17.425011 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.424950 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2mz8\" (UniqueName: \"kubernetes.io/projected/83f7a007-f5b5-4342-afac-4c971e6b413e-kube-api-access-x2mz8\") pod \"postgres-69fb4594cf-zlh92\" (UID: \"83f7a007-f5b5-4342-afac-4c971e6b413e\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:17.425091 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.425036 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/83f7a007-f5b5-4342-afac-4c971e6b413e-data\") pod \"postgres-69fb4594cf-zlh92\" (UID: \"83f7a007-f5b5-4342-afac-4c971e6b413e\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:17.425377 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.425361 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/83f7a007-f5b5-4342-afac-4c971e6b413e-data\") pod \"postgres-69fb4594cf-zlh92\" (UID: \"83f7a007-f5b5-4342-afac-4c971e6b413e\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:17.434245 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.434225 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2mz8\" (UniqueName: \"kubernetes.io/projected/83f7a007-f5b5-4342-afac-4c971e6b413e-kube-api-access-x2mz8\") pod \"postgres-69fb4594cf-zlh92\" (UID: \"83f7a007-f5b5-4342-afac-4c971e6b413e\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:17.475636 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.475619 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:17.610527 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.610499 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92"] Jun 10 15:14:17.612059 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:14:17.612032 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f7a007_f5b5_4342_afac_4c971e6b413e.slice/crio-ac783cbfd5508e569bcb13f4681b2d31e9d4e9cff7536284d63859196f5487f4 WatchSource:0}: Error finding container ac783cbfd5508e569bcb13f4681b2d31e9d4e9cff7536284d63859196f5487f4: Status 404 returned error can't find the container with id ac783cbfd5508e569bcb13f4681b2d31e9d4e9cff7536284d63859196f5487f4 Jun 10 15:14:17.675064 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:17.675003 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" event={"ID":"83f7a007-f5b5-4342-afac-4c971e6b413e","Type":"ContainerStarted","Data":"ac783cbfd5508e569bcb13f4681b2d31e9d4e9cff7536284d63859196f5487f4"} Jun 10 15:14:20.696281 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:20.696186 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" event={"ID":"83f7a007-f5b5-4342-afac-4c971e6b413e","Type":"ContainerStarted","Data":"57eeab4f779eec6fc309e2eb83b396955f2627c37475920dfdd2d56e3eab7558"} Jun 10 15:14:20.696281 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:20.696269 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:20.717253 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:20.717207 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" podStartSLOduration=0.916182336 podStartE2EDuration="3.717192027s" totalImagesPullingTime="2.801009691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:14:20.71489054 +0000 UTC m=+763.847183335" watchObservedRunningTime="2026-06-10 15:14:20.717192027 +0000 UTC m=+763.849484821" Jun 10 15:14:26.731090 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:26.731066 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-zlh92" Jun 10 15:14:27.037727 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:27.037641 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:14:27.136999 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:27.136967 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:14:27.309967 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:27.309915 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:14:28.467200 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.467163 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2"] Jun 10 15:14:28.471978 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.471956 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.474440 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.474417 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 15:14:28.475110 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.475091 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6nmjw\"" Jun 10 15:14:28.475110 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.475108 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 15:14:28.481954 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.481913 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2"] Jun 10 15:14:28.619474 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.619450 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.619580 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.619490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvqb4\" (UniqueName: \"kubernetes.io/projected/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-kube-api-access-qvqb4\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.619580 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.619516 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.719971 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.719910 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.720063 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.719982 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvqb4\" (UniqueName: \"kubernetes.io/projected/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-kube-api-access-qvqb4\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.720063 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.720019 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.720393 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.720372 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.720435 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.720398 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.732976 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.732954 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvqb4\" (UniqueName: \"kubernetes.io/projected/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-kube-api-access-qvqb4\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:28.781195 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:28.781175 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:29.125354 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:29.125297 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2"] Jun 10 15:14:29.127627 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:14:29.127601 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacef240b_8f8d_4d4f_8f74_fd401d0a81c4.slice/crio-fdcd0bfdb205d05666e14717e5ee4c64aeb077c62b34e50fe51baf4c15ee57f6 WatchSource:0}: Error finding container fdcd0bfdb205d05666e14717e5ee4c64aeb077c62b34e50fe51baf4c15ee57f6: Status 404 returned error can't find the container with id fdcd0bfdb205d05666e14717e5ee4c64aeb077c62b34e50fe51baf4c15ee57f6 Jun 10 15:14:29.750450 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:29.750415 2561 generic.go:356] "Generic (PLEG): container finished" podID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerID="1e376de7e54b77f2e9c15bb734dbc810cb1146d694f3d10216ef2cd263f7e3a8" exitCode=0 Jun 10 15:14:29.750857 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:29.750453 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" event={"ID":"acef240b-8f8d-4d4f-8f74-fd401d0a81c4","Type":"ContainerDied","Data":"1e376de7e54b77f2e9c15bb734dbc810cb1146d694f3d10216ef2cd263f7e3a8"} Jun 10 15:14:29.750857 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:29.750490 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" event={"ID":"acef240b-8f8d-4d4f-8f74-fd401d0a81c4","Type":"ContainerStarted","Data":"fdcd0bfdb205d05666e14717e5ee4c64aeb077c62b34e50fe51baf4c15ee57f6"} Jun 10 15:14:30.757904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:30.757877 2561 generic.go:356] "Generic (PLEG): container finished" podID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerID="ef97d027b93b933fd39db458471ea2c3fdffa579faa9692b204aaa3eeef0577e" exitCode=0 Jun 10 15:14:30.758243 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:30.757962 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" event={"ID":"acef240b-8f8d-4d4f-8f74-fd401d0a81c4","Type":"ContainerDied","Data":"ef97d027b93b933fd39db458471ea2c3fdffa579faa9692b204aaa3eeef0577e"} Jun 10 15:14:31.768903 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:31.768874 2561 generic.go:356] "Generic (PLEG): container finished" podID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerID="191b9a6c45d0891384c31703464e20e8b4c1862d61dc80c95738ce1d00c1adf1" exitCode=0 Jun 10 15:14:31.769281 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:31.768917 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" event={"ID":"acef240b-8f8d-4d4f-8f74-fd401d0a81c4","Type":"ContainerDied","Data":"191b9a6c45d0891384c31703464e20e8b4c1862d61dc80c95738ce1d00c1adf1"} Jun 10 15:14:32.916780 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:32.916760 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:32.961557 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:32.961531 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-bundle\") pod \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " Jun 10 15:14:32.961687 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:32.961573 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvqb4\" (UniqueName: \"kubernetes.io/projected/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-kube-api-access-qvqb4\") pod \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " Jun 10 15:14:32.961687 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:32.961618 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-util\") pod \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\" (UID: \"acef240b-8f8d-4d4f-8f74-fd401d0a81c4\") " Jun 10 15:14:32.962216 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:32.962193 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-bundle" (OuterVolumeSpecName: "bundle") pod "acef240b-8f8d-4d4f-8f74-fd401d0a81c4" (UID: "acef240b-8f8d-4d4f-8f74-fd401d0a81c4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:14:32.963760 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:32.963733 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-kube-api-access-qvqb4" (OuterVolumeSpecName: "kube-api-access-qvqb4") pod "acef240b-8f8d-4d4f-8f74-fd401d0a81c4" (UID: "acef240b-8f8d-4d4f-8f74-fd401d0a81c4"). InnerVolumeSpecName "kube-api-access-qvqb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:14:32.967207 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:32.967183 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-util" (OuterVolumeSpecName: "util") pod "acef240b-8f8d-4d4f-8f74-fd401d0a81c4" (UID: "acef240b-8f8d-4d4f-8f74-fd401d0a81c4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 15:14:33.062300 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:33.062278 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-util\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:14:33.062300 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:33.062301 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:14:33.062433 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:33.062310 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvqb4\" (UniqueName: \"kubernetes.io/projected/acef240b-8f8d-4d4f-8f74-fd401d0a81c4-kube-api-access-qvqb4\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:14:33.783344 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:33.783244 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" event={"ID":"acef240b-8f8d-4d4f-8f74-fd401d0a81c4","Type":"ContainerDied","Data":"fdcd0bfdb205d05666e14717e5ee4c64aeb077c62b34e50fe51baf4c15ee57f6"} Jun 10 15:14:33.783344 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:33.783266 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7484cng2" Jun 10 15:14:33.783344 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:33.783292 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdcd0bfdb205d05666e14717e5ee4c64aeb077c62b34e50fe51baf4c15ee57f6" Jun 10 15:14:36.489273 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.489242 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-dntr6"] Jun 10 15:14:36.489653 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.489640 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerName="extract" Jun 10 15:14:36.489698 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.489655 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerName="extract" Jun 10 15:14:36.489698 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.489695 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerName="util" Jun 10 15:14:36.489771 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.489702 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerName="util" Jun 10 15:14:36.489771 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.489708 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerName="pull" Jun 10 15:14:36.489771 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.489713 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerName="pull" Jun 10 15:14:36.489863 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.489771 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="acef240b-8f8d-4d4f-8f74-fd401d0a81c4" containerName="extract" Jun 10 15:14:36.492247 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.492230 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-dntr6" Jun 10 15:14:36.494813 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.494790 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 10 15:14:36.494951 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.494838 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 10 15:14:36.495646 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.495631 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-9fmqg\"" Jun 10 15:14:36.504831 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.504811 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-dntr6"] Jun 10 15:14:36.592904 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.592877 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h2bk\" (UniqueName: \"kubernetes.io/projected/0e0a1f65-bce4-4a91-ba83-fde22c4f8704-kube-api-access-9h2bk\") pod \"keycloak-operator-d67dbbb48-dntr6\" (UID: \"0e0a1f65-bce4-4a91-ba83-fde22c4f8704\") " pod="keycloak-system/keycloak-operator-d67dbbb48-dntr6" Jun 10 15:14:36.693335 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.693309 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h2bk\" (UniqueName: \"kubernetes.io/projected/0e0a1f65-bce4-4a91-ba83-fde22c4f8704-kube-api-access-9h2bk\") pod \"keycloak-operator-d67dbbb48-dntr6\" (UID: \"0e0a1f65-bce4-4a91-ba83-fde22c4f8704\") " pod="keycloak-system/keycloak-operator-d67dbbb48-dntr6" Jun 10 15:14:36.703187 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.703165 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h2bk\" (UniqueName: \"kubernetes.io/projected/0e0a1f65-bce4-4a91-ba83-fde22c4f8704-kube-api-access-9h2bk\") pod \"keycloak-operator-d67dbbb48-dntr6\" (UID: \"0e0a1f65-bce4-4a91-ba83-fde22c4f8704\") " pod="keycloak-system/keycloak-operator-d67dbbb48-dntr6" Jun 10 15:14:36.801829 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.801805 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-dntr6" Jun 10 15:14:36.940618 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:36.940590 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-dntr6"] Jun 10 15:14:36.941896 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:14:36.941871 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0a1f65_bce4_4a91_ba83_fde22c4f8704.slice/crio-dee09fdf20ea5d3c1b7eb4f712efdf10ddece72116acf17a46f1f49fa924aa8d WatchSource:0}: Error finding container dee09fdf20ea5d3c1b7eb4f712efdf10ddece72116acf17a46f1f49fa924aa8d: Status 404 returned error can't find the container with id dee09fdf20ea5d3c1b7eb4f712efdf10ddece72116acf17a46f1f49fa924aa8d Jun 10 15:14:37.809086 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:37.809048 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-dntr6" event={"ID":"0e0a1f65-bce4-4a91-ba83-fde22c4f8704","Type":"ContainerStarted","Data":"dee09fdf20ea5d3c1b7eb4f712efdf10ddece72116acf17a46f1f49fa924aa8d"} Jun 10 15:14:45.865606 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:45.865528 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-dntr6" event={"ID":"0e0a1f65-bce4-4a91-ba83-fde22c4f8704","Type":"ContainerStarted","Data":"7bbc575fc22c27a35e522e4aeb31f1f964cdd3702b1a114f7ac2eba19f19674d"} Jun 10 15:14:45.891372 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:45.891328 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d67dbbb48-dntr6" podStartSLOduration=1.8983942539999998 podStartE2EDuration="9.891312443s" totalImagesPullingTime="7.992918189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 15:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:14:45.890029146 +0000 UTC m=+789.022321939" watchObservedRunningTime="2026-06-10 15:14:45.891312443 +0000 UTC m=+789.023605239" Jun 10 15:14:53.948409 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:53.948371 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:14:54.047096 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:54.047059 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:14:59.474014 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:14:59.473985 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:24.482262 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:24.482229 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:24.578545 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:24.578520 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:30.685238 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:30.685204 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:30.805693 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:30.805658 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:30.916317 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:30.916292 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:31.062809 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:31.062785 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:34.228046 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:34.228013 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:34.340953 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:34.340896 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:49.648609 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:49.648578 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:15:49.719641 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:15:49.719610 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:16:06.798459 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:16:06.798426 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:16:06.966348 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:16:06.966323 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:16:07.379889 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:16:07.379856 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:16:07.487458 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:16:07.487429 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:16:49.571880 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:16:49.571851 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:16:49.671843 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:16:49.671799 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:16:57.958895 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:16:57.958857 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:16:59.192754 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:16:59.192726 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:17:00.509659 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:00.509625 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfd6bf448-46mz4"] Jun 10 15:17:00.641252 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:00.641205 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:17:00.734267 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:00.734225 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:17:00.801384 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:00.801356 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:17:11.170715 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:11.170684 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:17:25.534028 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.533968 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6dfd6bf448-46mz4" podUID="a1acddd3-68b4-48c1-8dfe-1e8217dfd039" containerName="console" containerID="cri-o://a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c" gracePeriod=15 Jun 10 15:17:25.790101 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.790079 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:17:25.814383 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.814255 2561 generic.go:356] "Generic (PLEG): container finished" podID="a1acddd3-68b4-48c1-8dfe-1e8217dfd039" containerID="a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c" exitCode=0 Jun 10 15:17:25.814538 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.814390 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfd6bf448-46mz4" event={"ID":"a1acddd3-68b4-48c1-8dfe-1e8217dfd039","Type":"ContainerDied","Data":"a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c"} Jun 10 15:17:25.814538 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.814440 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfd6bf448-46mz4" event={"ID":"a1acddd3-68b4-48c1-8dfe-1e8217dfd039","Type":"ContainerDied","Data":"2059f598cc476203b9a810f6b3803867e15248339d2a00a64323a84266f01b2e"} Jun 10 15:17:25.814538 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.814468 2561 scope.go:117] "RemoveContainer" containerID="a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c" Jun 10 15:17:25.814737 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.814610 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfd6bf448-46mz4" Jun 10 15:17:25.824351 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.824331 2561 scope.go:117] "RemoveContainer" containerID="a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c" Jun 10 15:17:25.824623 ip-10-0-129-85 kubenswrapper[2561]: E0610 15:17:25.824604 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c\": container with ID starting with a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c not found: ID does not exist" containerID="a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c" Jun 10 15:17:25.824677 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.824636 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c"} err="failed to get container status \"a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c\": rpc error: code = NotFound desc = could not find container \"a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c\": container with ID starting with a7117b681fe990a11d5870ccc8c03c585c7841979f058e823b11798adaff053c not found: ID does not exist" Jun 10 15:17:25.913429 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913394 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-serving-cert\") pod \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " Jun 10 15:17:25.913518 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913475 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-service-ca\") pod \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " Jun 10 15:17:25.913518 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913496 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-oauth-serving-cert\") pod \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " Jun 10 15:17:25.913643 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913535 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77bqk\" (UniqueName: \"kubernetes.io/projected/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-kube-api-access-77bqk\") pod \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " Jun 10 15:17:25.913643 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913560 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-oauth-config\") pod \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " Jun 10 15:17:25.913643 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913581 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-trusted-ca-bundle\") pod \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " Jun 10 15:17:25.913643 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913631 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-config\") pod \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\" (UID: \"a1acddd3-68b4-48c1-8dfe-1e8217dfd039\") " Jun 10 15:17:25.913996 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913919 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a1acddd3-68b4-48c1-8dfe-1e8217dfd039" (UID: "a1acddd3-68b4-48c1-8dfe-1e8217dfd039"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:17:25.913996 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913952 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-service-ca" (OuterVolumeSpecName: "service-ca") pod "a1acddd3-68b4-48c1-8dfe-1e8217dfd039" (UID: "a1acddd3-68b4-48c1-8dfe-1e8217dfd039"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:17:25.913996 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.913969 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a1acddd3-68b4-48c1-8dfe-1e8217dfd039" (UID: "a1acddd3-68b4-48c1-8dfe-1e8217dfd039"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:17:25.914204 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.914038 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-config" (OuterVolumeSpecName: "console-config") pod "a1acddd3-68b4-48c1-8dfe-1e8217dfd039" (UID: "a1acddd3-68b4-48c1-8dfe-1e8217dfd039"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 15:17:25.915408 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.915382 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a1acddd3-68b4-48c1-8dfe-1e8217dfd039" (UID: "a1acddd3-68b4-48c1-8dfe-1e8217dfd039"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 15:17:25.915686 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.915665 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a1acddd3-68b4-48c1-8dfe-1e8217dfd039" (UID: "a1acddd3-68b4-48c1-8dfe-1e8217dfd039"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 15:17:25.915835 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:25.915812 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-kube-api-access-77bqk" (OuterVolumeSpecName: "kube-api-access-77bqk") pod "a1acddd3-68b4-48c1-8dfe-1e8217dfd039" (UID: "a1acddd3-68b4-48c1-8dfe-1e8217dfd039"). InnerVolumeSpecName "kube-api-access-77bqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 15:17:26.014227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:26.014201 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-service-ca\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:17:26.014227 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:26.014226 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-oauth-serving-cert\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:17:26.014371 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:26.014241 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77bqk\" (UniqueName: \"kubernetes.io/projected/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-kube-api-access-77bqk\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:17:26.014371 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:26.014254 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-oauth-config\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:17:26.014371 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:26.014267 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-trusted-ca-bundle\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:17:26.014371 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:26.014281 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-config\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:17:26.014371 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:26.014295 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1acddd3-68b4-48c1-8dfe-1e8217dfd039-console-serving-cert\") on node \"ip-10-0-129-85.ec2.internal\" DevicePath \"\"" Jun 10 15:17:26.139608 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:26.139581 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfd6bf448-46mz4"] Jun 10 15:17:26.145512 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:26.145492 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dfd6bf448-46mz4"] Jun 10 15:17:27.350164 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:27.350131 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1acddd3-68b4-48c1-8dfe-1e8217dfd039" path="/var/lib/kubelet/pods/a1acddd3-68b4-48c1-8dfe-1e8217dfd039/volumes" Jun 10 15:17:27.627336 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:27.627249 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:17:38.773415 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:38.773378 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:17:46.656474 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:46.656394 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:17:57.112107 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:57.112050 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:17:59.986453 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:17:59.986422 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:18:12.892402 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:18:12.892370 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:19:29.660139 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:19:29.660101 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:19:33.185761 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:19:33.185732 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:19:49.953216 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:19:49.953185 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:19:57.016228 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:19:57.016194 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:24:58.179753 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:58.179723 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6f5db44995-7w9cb_65d70a14-227e-4142-837b-17fac5cfa766/nginx/0.log" Jun 10 15:24:58.293420 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:58.293386 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-5z7pm_a0d33a59-7ce9-4757-adf6-2271f8262bfc/manager/0.log" Jun 10 15:24:58.411438 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:58.411415 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-zsddd_e0158ddf-8fcb-4088-b35e-85167eba1630/manager/0.log" Jun 10 15:24:58.645992 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:58.645970 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-ppxll_e4e30e07-7b00-40cb-99d3-e429f480fd68/server/0.log" Jun 10 15:24:58.764486 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:58.764462 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-659rm_85893829-8aa2-4182-9726-0a6b4a3cef3f/manager/0.log" Jun 10 15:24:59.020201 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:59.020137 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58876c898f-mh7ct_61585677-1cfc-4feb-802c-010c1d873c2a/copy-manifests/0.log" Jun 10 15:24:59.033095 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:59.033074 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58876c898f-mh7ct_61585677-1cfc-4feb-802c-010c1d873c2a/manager/0.log" Jun 10 15:24:59.947849 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:59.947816 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6_bc9c3372-90f7-4ebf-b9e6-27112eb7350a/util/0.log" Jun 10 15:24:59.956181 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:59.956152 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6_bc9c3372-90f7-4ebf-b9e6-27112eb7350a/pull/0.log" Jun 10 15:24:59.965113 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:24:59.965094 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6_bc9c3372-90f7-4ebf-b9e6-27112eb7350a/extract/0.log" Jun 10 15:25:00.098527 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:00.098506 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf_a4248e78-61d4-44a2-945d-6b15352df733/util/0.log" Jun 10 15:25:00.114811 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:00.114793 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf_a4248e78-61d4-44a2-945d-6b15352df733/pull/0.log" Jun 10 15:25:00.130204 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:00.130184 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf_a4248e78-61d4-44a2-945d-6b15352df733/extract/0.log" Jun 10 15:25:00.249142 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:00.249080 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp_291116fb-12cf-4feb-b3aa-7f62565a3084/util/0.log" Jun 10 15:25:00.261094 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:00.261072 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp_291116fb-12cf-4feb-b3aa-7f62565a3084/pull/0.log" Jun 10 15:25:00.271813 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:00.271794 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp_291116fb-12cf-4feb-b3aa-7f62565a3084/extract/0.log" Jun 10 15:25:00.386616 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:00.386585 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w_75a5b9b4-5212-47da-a193-52d4cfcf8aa6/util/0.log" Jun 10 15:25:00.394989 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:00.394971 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w_75a5b9b4-5212-47da-a193-52d4cfcf8aa6/pull/0.log" Jun 10 15:25:00.403435 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:00.403418 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w_75a5b9b4-5212-47da-a193-52d4cfcf8aa6/extract/0.log" Jun 10 15:25:01.161431 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:01.161399 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hwf5q_9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d/manager/0.log" Jun 10 15:25:01.867641 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:01.867614 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-rpbbx_c216b24b-bef9-48d8-8c53-3bfe53ba5e7e/discovery/0.log" Jun 10 15:25:02.099007 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:02.098980 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b997c8484-lbblx_9257b5dc-9c61-4254-abca-f1650c82ee87/kube-auth-proxy/0.log" Jun 10 15:25:02.554560 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:02.554536 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c55898649-2nnlx_10bad15f-1514-446e-980c-33b2a8b5ca02/router/0.log" Jun 10 15:25:07.525426 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:07.525388 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:25:08.705476 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:08.705439 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:25:10.952323 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:10.952293 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fxmn5_c4b47316-b098-43dc-aa43-973bca48c3c2/global-pull-secret-syncer/0.log" Jun 10 15:25:11.123041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:11.123016 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" Jun 10 15:25:11.216580 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:11.216526 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-85.ec2.internal_bbfeb1994c2bac4a8dfa92f1e1823c15/haproxy/0.log" Jun 10 15:25:14.445923 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:14.445887 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-mvqsk_2e8ca265-d5d8-43ad-9fce-f07ff1aff584/sail-operator/0.log" Jun 10 15:25:15.962544 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:15.962464 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6_bc9c3372-90f7-4ebf-b9e6-27112eb7350a/extract/0.log" Jun 10 15:25:15.992279 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:15.992255 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6_bc9c3372-90f7-4ebf-b9e6-27112eb7350a/util/0.log" Jun 10 15:25:16.029329 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.029302 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjcb6_bc9c3372-90f7-4ebf-b9e6-27112eb7350a/pull/0.log" Jun 10 15:25:16.061791 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.061765 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf_a4248e78-61d4-44a2-945d-6b15352df733/extract/0.log" Jun 10 15:25:16.090498 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.090478 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf_a4248e78-61d4-44a2-945d-6b15352df733/util/0.log" Jun 10 15:25:16.116748 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.116728 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hxxf_a4248e78-61d4-44a2-945d-6b15352df733/pull/0.log" Jun 10 15:25:16.156875 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.156854 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp_291116fb-12cf-4feb-b3aa-7f62565a3084/extract/0.log" Jun 10 15:25:16.188741 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.188720 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp_291116fb-12cf-4feb-b3aa-7f62565a3084/util/0.log" Jun 10 15:25:16.217079 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.217028 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73k7kgp_291116fb-12cf-4feb-b3aa-7f62565a3084/pull/0.log" Jun 10 15:25:16.252664 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.252646 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w_75a5b9b4-5212-47da-a193-52d4cfcf8aa6/extract/0.log" Jun 10 15:25:16.298726 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.298701 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w_75a5b9b4-5212-47da-a193-52d4cfcf8aa6/util/0.log" Jun 10 15:25:16.319150 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.319128 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zl86w_75a5b9b4-5212-47da-a193-52d4cfcf8aa6/pull/0.log" Jun 10 15:25:16.566461 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:16.566433 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hwf5q_9c97b7e1-7ce3-4bc0-b932-ff4eadb5031d/manager/0.log" Jun 10 15:25:18.408437 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:18.408412 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-d67p5_516ba81a-80cd-49fe-bf46-f1eb09dae80b/cluster-monitoring-operator/0.log" Jun 10 15:25:18.614476 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:18.614411 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-szx86_c62d43e1-bd5b-48a0-951e-73693f36dedf/monitoring-plugin/0.log" Jun 10 15:25:18.659414 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:18.659394 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9jwp6_adf46b0f-5548-456d-8980-d56dd6ecb734/node-exporter/0.log" Jun 10 15:25:18.692795 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:18.692774 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9jwp6_adf46b0f-5548-456d-8980-d56dd6ecb734/kube-rbac-proxy/0.log" Jun 10 15:25:18.733028 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:18.733009 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9jwp6_adf46b0f-5548-456d-8980-d56dd6ecb734/init-textfile/0.log" Jun 10 15:25:18.977762 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:18.977710 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-bjq9c_1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf/kube-rbac-proxy-main/0.log" Jun 10 15:25:19.014384 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:19.014368 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-bjq9c_1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf/kube-rbac-proxy-self/0.log" Jun 10 15:25:19.050218 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:19.050199 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-bjq9c_1ccfa0d7-d2cb-40e5-8cbd-b1075ade2fdf/openshift-state-metrics/0.log" Jun 10 15:25:19.404631 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:19.404612 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-tvwzl_b0f56f66-1cbf-40e8-bea8-edaa9473f4c7/prometheus-operator-admission-webhook/0.log" Jun 10 15:25:19.566878 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:19.566856 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/thanos-query/0.log" Jun 10 15:25:19.601547 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:19.601523 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/kube-rbac-proxy-web/0.log" Jun 10 15:25:19.630041 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:19.630023 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/kube-rbac-proxy/0.log" Jun 10 15:25:19.668605 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:19.668556 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/prom-label-proxy/0.log" Jun 10 15:25:19.702815 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:19.702794 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/kube-rbac-proxy-rules/0.log" Jun 10 15:25:19.733358 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:19.733341 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c97668995-dwfmm_3a00bda7-20f0-4410-bed3-05190508710d/kube-rbac-proxy-metrics/0.log" Jun 10 15:25:20.952171 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:20.952138 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-c8w7n_75ab486a-c093-4fb3-bc99-0174f164ad3d/networking-console-plugin/0.log" Jun 10 15:25:21.556522 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.556494 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd"] Jun 10 15:25:21.556971 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.556953 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1acddd3-68b4-48c1-8dfe-1e8217dfd039" containerName="console" Jun 10 15:25:21.557053 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.556974 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1acddd3-68b4-48c1-8dfe-1e8217dfd039" containerName="console" Jun 10 15:25:21.557053 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.557047 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1acddd3-68b4-48c1-8dfe-1e8217dfd039" containerName="console" Jun 10 15:25:21.559348 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.559331 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.561773 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.561752 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mpsmf\"/\"openshift-service-ca.crt\"" Jun 10 15:25:21.561898 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.561757 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mpsmf\"/\"kube-root-ca.crt\"" Jun 10 15:25:21.562650 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.562633 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mpsmf\"/\"default-dockercfg-btzpr\"" Jun 10 15:25:21.572511 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.572489 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd"] Jun 10 15:25:21.707210 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.707188 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-lib-modules\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.707210 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.707213 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-podres\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.707373 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.707235 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-sys\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.707373 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.707288 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-proc\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.707373 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.707330 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7ddm\" (UniqueName: \"kubernetes.io/projected/fb506033-7d78-4373-9251-802c04fa0bba-kube-api-access-q7ddm\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.807858 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.807786 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-lib-modules\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.807858 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.807820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-podres\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.807858 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.807843 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-sys\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.807858 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.807865 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-proc\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.808130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.807925 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-proc\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.808130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.807956 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-sys\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.808130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.807978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7ddm\" (UniqueName: \"kubernetes.io/projected/fb506033-7d78-4373-9251-802c04fa0bba-kube-api-access-q7ddm\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.808130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.807982 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-podres\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.808130 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.807982 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb506033-7d78-4373-9251-802c04fa0bba-lib-modules\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.817552 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.817531 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7ddm\" (UniqueName: \"kubernetes.io/projected/fb506033-7d78-4373-9251-802c04fa0bba-kube-api-access-q7ddm\") pod \"perf-node-gather-daemonset-b66wd\" (UID: \"fb506033-7d78-4373-9251-802c04fa0bba\") " pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.870570 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.870548 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:21.953027 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:21.952962 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c89rn_91cc19fa-ee08-4fb7-bd80-84897bff8842/konnectivity-agent/0.log" limit=1024 Jun 10 15:25:22.002647 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:22.002619 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd"] Jun 10 15:25:22.003847 ip-10-0-129-85 kubenswrapper[2561]: W0610 15:25:22.003821 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfb506033_7d78_4373_9251_802c04fa0bba.slice/crio-537b06b43b73227171ae81445729d66ee9f8ffd149122dff00411e93cc73433e WatchSource:0}: Error finding container 537b06b43b73227171ae81445729d66ee9f8ffd149122dff00411e93cc73433e: Status 404 returned error can't find the container with id 537b06b43b73227171ae81445729d66ee9f8ffd149122dff00411e93cc73433e Jun 10 15:25:22.338307 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:22.338223 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-bjzrv_0f451555-80a8-4b4b-a876-9b49611345f6/download-server/0.log" Jun 10 15:25:22.613831 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:22.613758 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" event={"ID":"fb506033-7d78-4373-9251-802c04fa0bba","Type":"ContainerStarted","Data":"b5019384af6e281f84646e225bb0ea31dee4e670d419c2c81a52e4e566eee568"} Jun 10 15:25:22.613831 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:22.613827 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:22.614018 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:22.613854 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" event={"ID":"fb506033-7d78-4373-9251-802c04fa0bba","Type":"ContainerStarted","Data":"537b06b43b73227171ae81445729d66ee9f8ffd149122dff00411e93cc73433e"} Jun 10 15:25:22.633541 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:22.633500 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" podStartSLOduration=1.633484915 podStartE2EDuration="1.633484915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 15:25:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 15:25:22.632879179 +0000 UTC m=+1425.765171972" watchObservedRunningTime="2026-06-10 15:25:22.633484915 +0000 UTC m=+1425.765777706" Jun 10 15:25:23.018265 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:23.018194 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-jrfwf_53a0dcee-a209-4ffb-8f8c-a30209a86b19/volume-data-source-validator/0.log" Jun 10 15:25:24.356567 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:24.356531 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nh957_ad836858-b9dc-4287-b3e8-907485054bf6/dns/0.log" Jun 10 15:25:24.392293 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:24.392273 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nh957_ad836858-b9dc-4287-b3e8-907485054bf6/kube-rbac-proxy/0.log" Jun 10 15:25:24.468978 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:24.468958 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-djm7n_e512dd9b-89b7-4bb0-abef-1d717f999ddc/dns-node-resolver/0.log" Jun 10 15:25:25.215415 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:25.215375 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kvf47_49563c45-0b4c-404d-b5fd-46b64abfa6a8/node-ca/0.log" Jun 10 15:25:26.367462 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:26.367421 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-rpbbx_c216b24b-bef9-48d8-8c53-3bfe53ba5e7e/discovery/0.log" Jun 10 15:25:26.423371 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:26.423339 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b997c8484-lbblx_9257b5dc-9c61-4254-abca-f1650c82ee87/kube-auth-proxy/0.log" Jun 10 15:25:26.556267 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:26.556241 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c55898649-2nnlx_10bad15f-1514-446e-980c-33b2a8b5ca02/router/0.log" Jun 10 15:25:27.212751 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:27.212719 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cm2vf_a0531c81-3fe1-4f31-b45a-7aab9889966b/serve-healthcheck-canary/0.log" Jun 10 15:25:28.152065 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:28.152034 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vws5z_012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4/kube-rbac-proxy/0.log" Jun 10 15:25:28.186514 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:28.186491 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vws5z_012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4/exporter/0.log" Jun 10 15:25:28.217342 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:28.217316 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vws5z_012ed0c3-d0cf-4704-a8f6-7d6d5f83a2c4/extractor/0.log" Jun 10 15:25:28.631312 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:28.631284 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mpsmf/perf-node-gather-daemonset-b66wd" Jun 10 15:25:30.869256 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:30.869234 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6f5db44995-7w9cb_65d70a14-227e-4142-837b-17fac5cfa766/nginx/0.log" Jun 10 15:25:30.902016 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:30.901995 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-5z7pm_a0d33a59-7ce9-4757-adf6-2271f8262bfc/manager/0.log" Jun 10 15:25:30.943973 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:30.943953 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-zsddd_e0158ddf-8fcb-4088-b35e-85167eba1630/manager/0.log" Jun 10 15:25:31.004850 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:31.004822 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-ppxll_e4e30e07-7b00-40cb-99d3-e429f480fd68/server/0.log" Jun 10 15:25:31.061638 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:31.061618 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-659rm_85893829-8aa2-4182-9726-0a6b4a3cef3f/manager/0.log" Jun 10 15:25:31.185439 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:31.185387 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58876c898f-mh7ct_61585677-1cfc-4feb-802c-010c1d873c2a/manager/0.log" Jun 10 15:25:31.213392 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:31.213375 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58876c898f-mh7ct_61585677-1cfc-4feb-802c-010c1d873c2a/copy-manifests/0.log" Jun 10 15:25:35.616554 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:35.616523 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-rpbbx_c216b24b-bef9-48d8-8c53-3bfe53ba5e7e/discovery/0.log" Jun 10 15:25:35.736623 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:35.736600 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b997c8484-lbblx_9257b5dc-9c61-4254-abca-f1650c82ee87/kube-auth-proxy/0.log" Jun 10 15:25:35.836198 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:35.836174 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c55898649-2nnlx_10bad15f-1514-446e-980c-33b2a8b5ca02/router/0.log" Jun 10 15:25:36.995186 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:36.995159 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-rpbbx_c216b24b-bef9-48d8-8c53-3bfe53ba5e7e/discovery/0.log" Jun 10 15:25:37.024571 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:37.024551 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b997c8484-lbblx_9257b5dc-9c61-4254-abca-f1650c82ee87/kube-auth-proxy/0.log" Jun 10 15:25:37.111581 ip-10-0-129-85 kubenswrapper[2561]: I0610 15:25:37.111552 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c55898649-2nnlx_10bad15f-1514-446e-980c-33b2a8b5ca02/router/0.log"