Sep 18 21:05:57.238648 ip-10-0-129-163 systemd[1]: Starting Kubernetes Kubelet... Sep 18 21:05:57.641610 ip-10-0-129-163 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 21:05:57.641610 ip-10-0-129-163 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 18 21:05:57.641610 ip-10-0-129-163 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 21:05:57.641610 ip-10-0-129-163 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 18 21:05:57.641610 ip-10-0-129-163 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 21:05:57.642715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.642601 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 18 21:05:57.649016 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.648976 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 21:05:57.649016 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649008 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 21:05:57.649016 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649015 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 21:05:57.649016 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649021 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 21:05:57.649016 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649025 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 21:05:57.649016 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649028 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 21:05:57.649016 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649032 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 21:05:57.649016 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649035 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649039 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649043 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649046 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649049 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649052 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649054 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649057 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649060 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649063 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649065 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649068 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649070 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649073 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649077 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649079 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649082 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649085 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649088 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649090 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 21:05:57.649339 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649093 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649095 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649098 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649100 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649103 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649105 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649108 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649115 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649118 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649120 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649122 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649125 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649127 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649130 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649133 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649136 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649138 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649141 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649144 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649147 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 21:05:57.649852 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649152 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649155 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649158 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649161 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649164 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649167 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649169 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649172 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649175 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649177 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649180 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649182 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649185 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649188 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649190 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649193 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649195 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649198 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649201 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 21:05:57.650360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649204 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649206 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649209 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649211 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649214 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649216 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649219 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649222 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649225 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649228 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649230 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649233 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649235 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649238 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649241 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649244 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649246 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649249 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649252 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649255 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 21:05:57.650817 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649257 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649260 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649262 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649265 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649267 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649270 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649272 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649275 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649277 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649280 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649283 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649286 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.649289 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650009 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650016 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650019 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650022 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650025 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650027 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650031 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 21:05:57.651299 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650034 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650037 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650039 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650042 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650045 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650048 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650051 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650053 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650056 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650058 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650062 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650065 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650068 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650070 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650073 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650076 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650078 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650081 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650084 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650086 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 21:05:57.651797 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650089 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650092 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650094 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650097 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650100 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650103 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650106 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650108 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650111 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650113 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650116 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650119 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650123 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650127 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650130 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650133 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650136 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650138 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650141 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 21:05:57.652393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650143 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650147 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650151 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650153 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650156 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650159 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650162 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650165 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650167 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650170 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650173 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650175 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650178 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650180 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650182 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650185 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650188 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650191 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650194 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 21:05:57.652861 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650196 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650199 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650202 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650204 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650210 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650212 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650215 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650218 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650221 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650224 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650226 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650229 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650232 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650234 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650237 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650240 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650242 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650245 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650248 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 21:05:57.653369 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650251 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650253 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650256 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650259 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650262 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650264 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650267 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650269 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650273 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650277 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650280 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650283 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650287 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650289 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650292 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650698 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650703 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650709 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650711 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 21:05:57.653854 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650714 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650717 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650719 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650722 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650725 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650727 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650730 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650733 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650736 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650738 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650741 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650744 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650748 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650751 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650754 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650757 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650760 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650762 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650765 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 21:05:57.654326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650767 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650770 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650772 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650775 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650778 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650780 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650783 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650785 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650788 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650790 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650793 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650797 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650799 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650802 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650805 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650807 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650810 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650812 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650815 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650817 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 21:05:57.654827 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650820 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650829 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650832 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650834 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650837 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650840 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650843 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650846 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650849 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650852 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650854 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650857 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650859 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650862 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650868 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650871 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650875 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650878 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650881 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650883 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 21:05:57.655326 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650886 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650888 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650891 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650894 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650897 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650900 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650902 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650905 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650907 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650910 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650912 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650915 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650917 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650920 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650922 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650925 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650927 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650929 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650933 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650936 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 21:05:57.655826 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650938 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650941 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650945 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650948 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650950 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650954 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650956 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650959 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650962 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650964 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650967 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650969 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650972 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650974 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650976 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.650980 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651054 2557 flags.go:64] FLAG: --address="0.0.0.0" Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651089 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651096 2557 flags.go:64] FLAG: --anonymous-auth="true" Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651101 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 18 21:05:57.656388 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651105 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651108 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651112 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651116 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651119 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651122 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651126 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651129 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651132 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651135 2557 flags.go:64] FLAG: --cgroup-root="" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651138 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651141 2557 flags.go:64] FLAG: --client-ca-file="" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651144 2557 flags.go:64] FLAG: --cloud-provider="external" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651148 2557 flags.go:64] FLAG: --cluster-dns="[]" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651152 2557 flags.go:64] FLAG: --cluster-domain="" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651155 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651158 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651161 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651164 2557 flags.go:64] FLAG: --container-log-max-files="5" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651168 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651171 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651174 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651177 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651180 2557 flags.go:64] FLAG: --contention-profiling="false" Sep 18 21:05:57.656863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651183 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651186 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651189 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651193 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651197 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651200 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651203 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651206 2557 flags.go:64] FLAG: --enable-load-reader="false" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651209 2557 flags.go:64] FLAG: --enable-server="true" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651212 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651217 2557 flags.go:64] FLAG: --event-burst="100" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651220 2557 flags.go:64] FLAG: --event-qps="50" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651222 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651225 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651228 2557 flags.go:64] FLAG: --eviction-hard="" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651232 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651235 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651238 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651244 2557 flags.go:64] FLAG: --eviction-soft="" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651247 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651250 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651253 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651257 2557 flags.go:64] FLAG: --experimental-mounter-path="" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651260 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651263 2557 flags.go:64] FLAG: --fail-swap-on="true" Sep 18 21:05:57.657429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651266 2557 flags.go:64] FLAG: --feature-gates="" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651269 2557 flags.go:64] FLAG: --file-check-frequency="20s" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651272 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651275 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651278 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651281 2557 flags.go:64] FLAG: --healthz-port="10248" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651285 2557 flags.go:64] FLAG: --help="false" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651287 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-129-163.ec2.internal" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651291 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651294 2557 flags.go:64] FLAG: --http-check-frequency="20s" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651298 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651301 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651305 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651308 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651339 2557 flags.go:64] FLAG: --image-service-endpoint="" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651343 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651346 2557 flags.go:64] FLAG: --kube-api-burst="100" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651349 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651352 2557 flags.go:64] FLAG: --kube-api-qps="50" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651355 2557 flags.go:64] FLAG: --kube-reserved="" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651359 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651362 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651365 2557 flags.go:64] FLAG: --kubelet-cgroups="" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651367 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 18 21:05:57.658057 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651370 2557 flags.go:64] FLAG: --lock-file="" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651375 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651378 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651381 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651386 2557 flags.go:64] FLAG: --log-json-split-stream="false" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651390 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651393 2557 flags.go:64] FLAG: --log-text-split-stream="false" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651396 2557 flags.go:64] FLAG: --logging-format="text" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651399 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651402 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651405 2557 flags.go:64] FLAG: --manifest-url="" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651408 2557 flags.go:64] FLAG: --manifest-url-header="" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651413 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651416 2557 flags.go:64] FLAG: --max-open-files="1000000" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651426 2557 flags.go:64] FLAG: --max-pods="110" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651429 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651431 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651435 2557 flags.go:64] FLAG: --memory-manager-policy="None" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651438 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651441 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651444 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651447 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651455 2557 flags.go:64] FLAG: --node-status-max-images="50" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651458 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651461 2557 flags.go:64] FLAG: --oom-score-adj="-999" Sep 18 21:05:57.658691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651464 2557 flags.go:64] FLAG: --pod-cidr="" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651475 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651481 2557 flags.go:64] FLAG: --pod-manifest-path="" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651485 2557 flags.go:64] FLAG: --pod-max-pids="-1" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651488 2557 flags.go:64] FLAG: --pods-per-core="0" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651491 2557 flags.go:64] FLAG: --port="10250" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651494 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651497 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-050ae3fac38d3a0b7" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651500 2557 flags.go:64] FLAG: --qos-reserved="" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651505 2557 flags.go:64] FLAG: --read-only-port="10255" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651508 2557 flags.go:64] FLAG: --register-node="true" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651510 2557 flags.go:64] FLAG: --register-with-taints="" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651514 2557 flags.go:64] FLAG: --registry-burst="10" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651518 2557 flags.go:64] FLAG: --registry-qps="5" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651521 2557 flags.go:64] FLAG: --reserved-cpus="" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651524 2557 flags.go:64] FLAG: --reserved-memory="" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651527 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651530 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651533 2557 flags.go:64] FLAG: --rotate-certificates="false" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651536 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651539 2557 flags.go:64] FLAG: --runonce="false" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651542 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651545 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651548 2557 flags.go:64] FLAG: --seccomp-default="false" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651551 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 18 21:05:57.659303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651554 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651557 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651560 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651563 2557 flags.go:64] FLAG: --storage-driver-password="root" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651566 2557 flags.go:64] FLAG: --storage-driver-secure="false" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651569 2557 flags.go:64] FLAG: --storage-driver-table="stats" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651572 2557 flags.go:64] FLAG: --storage-driver-user="root" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651575 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651578 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651581 2557 flags.go:64] FLAG: --system-cgroups="" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651584 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651589 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651593 2557 flags.go:64] FLAG: --tls-cert-file="" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651596 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651600 2557 flags.go:64] FLAG: --tls-min-version="" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651602 2557 flags.go:64] FLAG: --tls-private-key-file="" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651607 2557 flags.go:64] FLAG: --topology-manager-policy="none" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651610 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651613 2557 flags.go:64] FLAG: --topology-manager-scope="container" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651616 2557 flags.go:64] FLAG: --v="2" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651620 2557 flags.go:64] FLAG: --version="false" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651624 2557 flags.go:64] FLAG: --vmodule="" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651628 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.651631 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651744 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 21:05:57.659967 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651747 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651751 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651754 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651757 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651760 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651763 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651765 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651768 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651770 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651773 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651776 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651779 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651782 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651785 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651787 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651790 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651793 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651795 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651798 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 21:05:57.660639 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651802 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651805 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651809 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651813 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651817 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651820 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651824 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651826 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651830 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651833 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651836 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651839 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651842 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651845 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651847 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651850 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651852 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651855 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651858 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 21:05:57.661102 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651860 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651863 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651865 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651868 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651871 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651875 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651878 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651881 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651884 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651886 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651889 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651892 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651894 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651897 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651900 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651903 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651905 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651909 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651912 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651915 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 21:05:57.661601 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651917 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651920 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651923 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651926 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651929 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651931 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651934 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651937 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651939 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651942 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651944 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651947 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651951 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651953 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651956 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651959 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651962 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651964 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651968 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651970 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 21:05:57.662090 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651973 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651975 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651978 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651981 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651983 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651986 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651988 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651991 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651993 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.651998 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652001 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652003 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652006 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652008 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652011 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652014 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652016 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652020 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652024 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 21:05:57.662604 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.652028 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.652781 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.659558 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.659577 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.660369 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.660376 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660440 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660445 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660448 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660452 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660456 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660460 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660464 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660467 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 21:05:57.663062 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660470 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660473 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660477 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660481 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660484 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660487 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660490 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660493 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660496 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660498 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660501 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660503 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660506 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660509 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660512 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660516 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660520 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660523 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660526 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660530 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 21:05:57.663574 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660532 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660535 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660538 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660541 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660543 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660546 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660548 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660551 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660553 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660556 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660558 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660561 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660564 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660566 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660569 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660571 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660574 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660576 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660579 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660582 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 21:05:57.664054 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660584 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660587 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660589 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660592 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660594 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660597 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660599 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660602 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660604 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660608 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660610 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660613 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660616 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660619 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660621 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660624 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660626 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660629 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660631 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660634 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 21:05:57.664545 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660637 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660639 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660642 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660645 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660647 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660650 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660653 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660656 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660658 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660661 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660664 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660666 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660669 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660671 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660673 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660676 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660679 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660681 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660684 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660687 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 21:05:57.665040 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660689 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660692 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660695 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660698 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660701 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660704 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660707 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660710 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660712 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660715 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660717 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.660722 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660835 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660840 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660843 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 21:05:57.665532 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660846 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660849 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660851 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660854 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660856 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660859 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660861 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660864 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660867 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660869 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660872 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660875 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660878 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660880 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660883 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660885 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660888 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660891 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660893 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660896 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 21:05:57.665899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660899 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660902 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660905 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660908 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660911 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660913 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660916 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660918 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660921 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660923 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660926 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660929 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660932 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660936 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660939 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660941 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660944 2557 feature_gate.go:328] unrecognized feature gate: Example Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660947 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660949 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 21:05:57.666403 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660952 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660954 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660957 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660959 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660962 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660964 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660967 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660970 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660972 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660975 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660977 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660980 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660982 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660985 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660990 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660993 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660996 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.660999 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661002 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 21:05:57.666858 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661005 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661008 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661011 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661013 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661016 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661019 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661022 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661024 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661027 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661030 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661032 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661035 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661037 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661040 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661042 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661045 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661049 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661052 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661056 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661058 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 21:05:57.667413 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661061 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661064 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661067 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661070 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661073 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661075 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661078 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661081 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661084 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661087 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661090 2557 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661092 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661095 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661097 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661100 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661102 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661105 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 21:05:57.667899 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:57.661108 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 21:05:57.668309 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.661113 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 21:05:57.668309 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.661225 2557 server.go:962] "Client rotation is on, will bootstrap in background" Sep 18 21:05:57.668309 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.663204 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 18 21:05:57.668309 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.664636 2557 server.go:1019] "Starting client certificate rotation" Sep 18 21:05:57.668309 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.664737 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 21:05:57.668309 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.665376 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 21:05:57.668309 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.667364 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 21:05:57.668555 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.668525 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 21:05:57.679489 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.679460 2557 log.go:25] "Validated CRI v1 runtime API" Sep 18 21:05:57.681132 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.681117 2557 log.go:25] "Validated CRI v1 image API" Sep 18 21:05:57.681984 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.681961 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 18 21:05:57.685890 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.685867 2557 fs.go:135] Filesystem UUIDs: map[0713c844-205e-4faf-90da-a4e3cf00478c:/dev/nvme0n1p3 5b5f09b1-1327-4161-963b-cac48529d250:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 18 21:05:57.685989 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.685889 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 18 21:05:57.691134 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.690893 2557 manager.go:216] Machine: {Timestamp:2026-09-18 21:05:57.689631187 +0000 UTC m=+0.338228769 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100396 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fa0ee369b12d7e51f09a2d063e7e9 SystemUUID:ec2fa0ee-369b-12d7-e51f-09a2d063e7e9 BootID:c1c86e41-ab71-4418-9771-e16033af448e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e4:03:a1:05 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e4:03:a1:05 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:d6:94:16:0a:fd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 18 21:05:57.691134 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.691131 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 18 21:05:57.691282 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.691214 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 18 21:05:57.693416 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.693391 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 18 21:05:57.693569 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.693420 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-163.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 18 21:05:57.694692 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.694682 2557 topology_manager.go:138] "Creating topology manager with none policy" Sep 18 21:05:57.694730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.694695 2557 container_manager_linux.go:309] "Creating device plugin manager" Sep 18 21:05:57.694730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.694704 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 21:05:57.695185 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.695170 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 21:05:57.695361 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.695350 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 21:05:57.696751 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.696740 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 18 21:05:57.699821 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.699809 2557 state_mem.go:36] "Initialized new in-memory state store" Sep 18 21:05:57.699936 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.699927 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 18 21:05:57.699965 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.699955 2557 kubelet.go:486] "Attempting to sync node with API server" Sep 18 21:05:57.700000 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.699964 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 18 21:05:57.700000 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.699981 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 18 21:05:57.700000 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.699990 2557 kubelet.go:398] "Adding apiserver pod source" Sep 18 21:05:57.700000 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.699999 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 18 21:05:57.700382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.700371 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 21:05:57.700438 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.700389 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 21:05:57.702574 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.702558 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 18 21:05:57.704719 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.704707 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 18 21:05:57.704762 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.704738 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 18 21:05:57.706245 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706231 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706249 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706256 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706262 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706271 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706276 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706290 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706296 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706302 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706308 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 18 21:05:57.706324 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706338 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 18 21:05:57.706590 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.706344 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 18 21:05:57.707048 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.707038 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 18 21:05:57.707048 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.707048 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 18 21:05:57.710798 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.710783 2557 server.go:1268] "Started kubelet" Sep 18 21:05:57.710909 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.710889 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-163.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 18 21:05:57.710983 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.710937 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 18 21:05:57.710983 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.710980 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 18 21:05:57.711081 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.710984 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 18 21:05:57.711081 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.711065 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 18 21:05:57.711331 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.711285 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 18 21:05:57.711761 ip-10-0-129-163 systemd[1]: Started Kubernetes Kubelet. Sep 18 21:05:57.712184 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.712165 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 18 21:05:57.712398 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.712386 2557 server.go:310] "Adding debug handlers to kubelet server" Sep 18 21:05:57.717171 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.717146 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 18 21:05:57.721067 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.720102 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-163.ec2.internal.18d6864aca943062 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-163.ec2.internal,UID:ip-10-0-129-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-163.ec2.internal,},FirstTimestamp:2026-09-18 21:05:57.710762082 +0000 UTC m=+0.359359664,LastTimestamp:2026-09-18 21:05:57.710762082 +0000 UTC m=+0.359359664,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-163.ec2.internal,}" Sep 18 21:05:57.721731 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.721710 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 18 21:05:57.722179 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.722162 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 18 21:05:57.722349 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.722330 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:57.722753 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.722733 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 18 21:05:57.722753 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.722752 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 18 21:05:57.722893 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.722733 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 18 21:05:57.722893 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.722869 2557 reconstruct.go:97] "Volume reconstruction finished" Sep 18 21:05:57.722893 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.722877 2557 reconciler.go:29] "Reconciler: start to sync state" Sep 18 21:05:57.723087 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.723073 2557 factory.go:55] Registering systemd factory Sep 18 21:05:57.723087 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.723088 2557 factory.go:223] Registration of the systemd container factory successfully Sep 18 21:05:57.723337 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.723307 2557 factory.go:153] Registering CRI-O factory Sep 18 21:05:57.723403 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.723341 2557 factory.go:223] Registration of the crio container factory successfully Sep 18 21:05:57.723403 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.723389 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 18 21:05:57.723484 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.723408 2557 factory.go:103] Registering Raw factory Sep 18 21:05:57.723484 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.723420 2557 manager.go:1196] Started watching for new ooms in manager Sep 18 21:05:57.723737 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.723726 2557 manager.go:349] Starting recovery of all containers Sep 18 21:05:57.733870 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.733843 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 18 21:05:57.734007 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.733962 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 18 21:05:57.734080 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.734068 2557 manager.go:354] Recovery completed Sep 18 21:05:57.739462 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.739452 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 21:05:57.741541 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.741509 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientMemory" Sep 18 21:05:57.741606 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.741560 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 21:05:57.741606 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.741574 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientPID" Sep 18 21:05:57.742113 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.742099 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 18 21:05:57.742113 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.742112 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 18 21:05:57.742214 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.742124 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 21:05:57.742214 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.742126 2557 state_mem.go:36] "Initialized new in-memory state store" Sep 18 21:05:57.742836 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.742818 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientMemory" Sep 18 21:05:57.742903 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.742849 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 21:05:57.742903 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.742861 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientPID" Sep 18 21:05:57.743366 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.743353 2557 policy_none.go:49] "None policy: Start" Sep 18 21:05:57.743411 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.743370 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 18 21:05:57.743411 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.743381 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 18 21:05:57.744355 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.744341 2557 policy_none.go:47] "Start" Sep 18 21:05:57.753382 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.753290 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-163.ec2.internal.18d6864acc69d880 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-163.ec2.internal,UID:ip-10-0-129-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-163.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-163.ec2.internal,},FirstTimestamp:2026-09-18 21:05:57.741541504 +0000 UTC m=+0.390139093,LastTimestamp:2026-09-18 21:05:57.741541504 +0000 UTC m=+0.390139093,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-163.ec2.internal,}" Sep 18 21:05:57.757814 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.757801 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lmwkt" Sep 18 21:05:57.764572 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.764549 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lmwkt" Sep 18 21:05:57.765072 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.764977 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-163.ec2.internal.18d6864acc6a3e69 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-163.ec2.internal,UID:ip-10-0-129-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-163.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-163.ec2.internal,},FirstTimestamp:2026-09-18 21:05:57.741567593 +0000 UTC m=+0.390165174,LastTimestamp:2026-09-18 21:05:57.741567593 +0000 UTC m=+0.390165174,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-163.ec2.internal,}" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.783838 2557 manager.go:337] "Starting Device Plugin manager" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.783882 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.783895 2557 server.go:87] "Starting device plugin registration server" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.784261 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.784272 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.784373 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.784510 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.784521 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.785121 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 18 21:05:57.790074 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.785166 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:57.852378 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.852346 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 18 21:05:57.853543 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.853520 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 18 21:05:57.853655 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.853576 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 18 21:05:57.853655 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.853600 2557 kubelet.go:2448] "Starting kubelet main sync loop" Sep 18 21:05:57.853655 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.853649 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 18 21:05:57.856380 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.856361 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 21:05:57.884697 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.884677 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 21:05:57.885670 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.885652 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientMemory" Sep 18 21:05:57.885758 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.885682 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 21:05:57.885758 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.885693 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientPID" Sep 18 21:05:57.885758 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.885720 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-163.ec2.internal" Sep 18 21:05:57.895906 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.895840 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-163.ec2.internal" Sep 18 21:05:57.895906 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.895864 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-163.ec2.internal\": node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:57.927590 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.927556 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:57.954528 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.954502 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal"] Sep 18 21:05:57.954678 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.954576 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 21:05:57.956060 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.956044 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientMemory" Sep 18 21:05:57.956148 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.956074 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 21:05:57.956148 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.956084 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientPID" Sep 18 21:05:57.957356 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.957344 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 21:05:57.957456 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.957441 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" Sep 18 21:05:57.957508 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.957479 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 21:05:57.958135 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.958118 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientMemory" Sep 18 21:05:57.958203 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.958147 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 21:05:57.958203 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.958155 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientMemory" Sep 18 21:05:57.958203 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.958174 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 21:05:57.958203 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.958187 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientPID" Sep 18 21:05:57.958203 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.958159 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientPID" Sep 18 21:05:57.959527 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.959512 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal" Sep 18 21:05:57.959589 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.959539 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 21:05:57.960338 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.960306 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientMemory" Sep 18 21:05:57.960437 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.960352 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 21:05:57.960437 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:57.960367 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientPID" Sep 18 21:05:57.984187 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.984163 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-163.ec2.internal\" not found" node="ip-10-0-129-163.ec2.internal" Sep 18 21:05:57.988707 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:57.988689 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-163.ec2.internal\" not found" node="ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.024421 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.024387 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d0288c42e3cdc1f6a95ea48965eccccf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal\" (UID: \"d0288c42e3cdc1f6a95ea48965eccccf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.024421 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.024421 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0288c42e3cdc1f6a95ea48965eccccf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal\" (UID: \"d0288c42e3cdc1f6a95ea48965eccccf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.024598 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.024439 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0177e4b7799939fec863354aec680a58-config\") pod \"kube-apiserver-proxy-ip-10-0-129-163.ec2.internal\" (UID: \"0177e4b7799939fec863354aec680a58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.028501 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.028484 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:58.125335 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.125285 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d0288c42e3cdc1f6a95ea48965eccccf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal\" (UID: \"d0288c42e3cdc1f6a95ea48965eccccf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.125496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.125339 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0288c42e3cdc1f6a95ea48965eccccf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal\" (UID: \"d0288c42e3cdc1f6a95ea48965eccccf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.125496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.125366 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0177e4b7799939fec863354aec680a58-config\") pod \"kube-apiserver-proxy-ip-10-0-129-163.ec2.internal\" (UID: \"0177e4b7799939fec863354aec680a58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.125496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.125426 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d0288c42e3cdc1f6a95ea48965eccccf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal\" (UID: \"d0288c42e3cdc1f6a95ea48965eccccf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.125496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.125437 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0177e4b7799939fec863354aec680a58-config\") pod \"kube-apiserver-proxy-ip-10-0-129-163.ec2.internal\" (UID: \"0177e4b7799939fec863354aec680a58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.125496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.125456 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0288c42e3cdc1f6a95ea48965eccccf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal\" (UID: \"d0288c42e3cdc1f6a95ea48965eccccf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.129398 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.129381 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:58.230232 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.230140 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:58.290288 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.290242 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.291859 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.291840 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal" Sep 18 21:05:58.330329 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.330281 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:58.431021 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.430979 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:58.531690 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.531620 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:58.631723 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.631685 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:58.663587 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.663556 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 18 21:05:58.664069 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.663737 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 18 21:05:58.732241 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.732209 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:58.769277 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.769219 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 21:00:58 +0000 UTC" deadline="2028-04-06 21:56:39.030666489 +0000 UTC" Sep 18 21:05:58.769277 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.769275 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13584h50m40.261397111s" Sep 18 21:05:58.833238 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.833167 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:58.878310 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:58.878092 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0288c42e3cdc1f6a95ea48965eccccf.slice/crio-47e32636a6b91619a1057b870d423731b5febec45340967b21d89eb26ae02bbb WatchSource:0}: Error finding container 47e32636a6b91619a1057b870d423731b5febec45340967b21d89eb26ae02bbb: Status 404 returned error can't find the container with id 47e32636a6b91619a1057b870d423731b5febec45340967b21d89eb26ae02bbb Sep 18 21:05:58.878642 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:05:58.878619 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0177e4b7799939fec863354aec680a58.slice/crio-c6ddb433c3ef96a69e7f3c66848d48a2a95b391aae2173ee6bbb21aabeb39046 WatchSource:0}: Error finding container c6ddb433c3ef96a69e7f3c66848d48a2a95b391aae2173ee6bbb21aabeb39046: Status 404 returned error can't find the container with id c6ddb433c3ef96a69e7f3c66848d48a2a95b391aae2173ee6bbb21aabeb39046 Sep 18 21:05:58.883651 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.883639 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 21:05:58.900097 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.900078 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 21:05:58.915056 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.915034 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 21:05:58.921479 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.921462 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 21:05:58.922577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.922551 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientMemory" Sep 18 21:05:58.922690 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.922595 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 21:05:58.922690 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:58.922611 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeHasSufficientPID" Sep 18 21:05:58.933909 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:58.933889 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:59.034752 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.034718 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-163.ec2.internal\" not found" Sep 18 21:05:59.115097 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.115013 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 21:05:59.122712 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.122688 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" Sep 18 21:05:59.137799 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.137772 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 21:05:59.138880 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.138866 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal" Sep 18 21:05:59.148724 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.148708 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 21:05:59.500750 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.500489 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 21:05:59.700917 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.700879 2557 apiserver.go:52] "Watching apiserver" Sep 18 21:05:59.710189 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.710156 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 18 21:05:59.711379 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.711332 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-6cmhj","openshift-network-operator/iptables-alerter-6nct6","kube-system/konnectivity-agent-jhzsp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk","openshift-cluster-node-tuning-operator/tuned-9jqg2","openshift-dns/node-resolver-pj68m","openshift-image-registry/node-ca-pvrc4","openshift-multus/multus-2pc68","openshift-multus/network-metrics-daemon-bvvml","openshift-ovn-kubernetes/ovnkube-node-fqsfb","kube-system/global-pull-secret-syncer-hrxgz","kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal","openshift-multus/multus-additional-cni-plugins-9sm5k"] Sep 18 21:05:59.712884 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.712856 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.714516 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.714494 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:05:59.714625 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.714573 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:05:59.715730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.715493 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 18 21:05:59.715730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.715598 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 18 21:05:59.715872 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.715844 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xcwwc\"" Sep 18 21:05:59.715872 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.715859 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:05:59.716031 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.716011 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 18 21:05:59.717038 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.717016 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.718595 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.718421 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 18 21:05:59.718769 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.718752 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.719384 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.719362 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-trvjb\"" Sep 18 21:05:59.719498 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.719448 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 18 21:05:59.719557 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.719522 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 18 21:05:59.719935 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.719912 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 18 21:05:59.720013 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.719963 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-49jnd\"" Sep 18 21:05:59.720067 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.720039 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 18 21:05:59.720768 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.720746 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.721308 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.721289 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 18 21:05:59.721461 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.721425 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mkgxx\"" Sep 18 21:05:59.722442 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.721762 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 18 21:05:59.722585 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.722566 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 18 21:05:59.723052 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.723033 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:05:59.723128 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.723108 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:05:59.724523 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.724501 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 18 21:05:59.724523 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.724518 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fmzx5\"" Sep 18 21:05:59.724655 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.724585 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 18 21:05:59.726218 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.726201 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.727585 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.727567 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.729717 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.729406 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.730089 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.730060 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 18 21:05:59.730186 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.730100 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 18 21:05:59.730186 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.730074 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6np4m\"" Sep 18 21:05:59.730298 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.730233 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 18 21:05:59.730364 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.730325 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 18 21:05:59.730911 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.730891 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 18 21:05:59.731366 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.731345 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 18 21:05:59.731507 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.731491 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:05:59.731572 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.731554 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:05:59.731640 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.731613 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 18 21:05:59.731860 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.731840 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-q2qz7\"" Sep 18 21:05:59.731962 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.731878 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 18 21:05:59.732157 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.732134 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 18 21:05:59.733286 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.733265 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 18 21:05:59.733397 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.733307 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.733755 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.733730 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 18 21:05:59.733863 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.733803 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j95k4\"" Sep 18 21:05:59.734586 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.734560 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 18 21:05:59.734682 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.734644 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 18 21:05:59.734892 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.734873 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 18 21:05:59.736562 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.736523 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvvfz\" (UniqueName: \"kubernetes.io/projected/133f1088-0c44-446d-832c-1d893f21ab04-kube-api-access-vvvfz\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.736658 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.736567 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:05:59.736658 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.736596 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cbba6619-7498-4ef7-a9a8-174ffc7c3953-kubelet-config\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:05:59.736658 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.736606 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 18 21:05:59.737475 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737454 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pqzn\" (UniqueName: \"kubernetes.io/projected/20959b5d-e329-49fe-8e03-617b788df45f-kube-api-access-8pqzn\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.737575 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737493 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-lib-modules\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.737575 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737512 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/133f1088-0c44-446d-832c-1d893f21ab04-etc-tuned\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.737575 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737527 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/133f1088-0c44-446d-832c-1d893f21ab04-tmp\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.737575 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737545 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.737575 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737569 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/626b4d06-d3b0-44fb-94b9-94f3131d6afc-hosts-file\") pod \"node-resolver-pj68m\" (UID: \"626b4d06-d3b0-44fb-94b9-94f3131d6afc\") " pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737599 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/597032d6-55fc-4ec1-a791-533a72693ad2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737621 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-kubernetes\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737639 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-host\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737655 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cbba6619-7498-4ef7-a9a8-174ffc7c3953-dbus\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/597032d6-55fc-4ec1-a791-533a72693ad2-system-cni-dir\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737702 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlkhl\" (UniqueName: \"kubernetes.io/projected/597032d6-55fc-4ec1-a791-533a72693ad2-kube-api-access-zlkhl\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737731 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-modprobe-d\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737752 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-var-lib-kubelet\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737776 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zw2hx\"" Sep 18 21:05:59.737831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737779 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737836 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/369e25ea-147c-4099-a089-03876f8f9ec5-agent-certs\") pod \"konnectivity-agent-jhzsp\" (UID: \"369e25ea-147c-4099-a089-03876f8f9ec5\") " pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737866 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-registration-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737891 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/626b4d06-d3b0-44fb-94b9-94f3131d6afc-tmp-dir\") pod \"node-resolver-pj68m\" (UID: \"626b4d06-d3b0-44fb-94b9-94f3131d6afc\") " pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737917 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbjh8\" (UniqueName: \"kubernetes.io/projected/626b4d06-d3b0-44fb-94b9-94f3131d6afc-kube-api-access-jbjh8\") pod \"node-resolver-pj68m\" (UID: \"626b4d06-d3b0-44fb-94b9-94f3131d6afc\") " pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737940 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28264d96-1e7c-49bf-9654-5ad9c675d9b6-host\") pod \"node-ca-pvrc4\" (UID: \"28264d96-1e7c-49bf-9654-5ad9c675d9b6\") " pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.737962 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-sysctl-d\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738058 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-sys\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738097 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/369e25ea-147c-4099-a089-03876f8f9ec5-konnectivity-ca\") pod \"konnectivity-agent-jhzsp\" (UID: \"369e25ea-147c-4099-a089-03876f8f9ec5\") " pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738126 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-socket-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738158 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-etc-selinux\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.738228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738208 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/597032d6-55fc-4ec1-a791-533a72693ad2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738247 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28264d96-1e7c-49bf-9654-5ad9c675d9b6-serviceca\") pod \"node-ca-pvrc4\" (UID: \"28264d96-1e7c-49bf-9654-5ad9c675d9b6\") " pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738282 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4bfd\" (UniqueName: \"kubernetes.io/projected/28264d96-1e7c-49bf-9654-5ad9c675d9b6-kube-api-access-x4bfd\") pod \"node-ca-pvrc4\" (UID: \"28264d96-1e7c-49bf-9654-5ad9c675d9b6\") " pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738345 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-device-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738373 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-sysconfig\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738400 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-sysctl-conf\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738426 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-systemd\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738453 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-run\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738470 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4r62\" (UniqueName: \"kubernetes.io/projected/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-kube-api-access-s4r62\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738500 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/597032d6-55fc-4ec1-a791-533a72693ad2-cni-binary-copy\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738523 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/597032d6-55fc-4ec1-a791-533a72693ad2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738541 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-sys-fs\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.738838 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.738582 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/597032d6-55fc-4ec1-a791-533a72693ad2-cnibin\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.752067 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.751610 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 21:05:59.768937 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.768912 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-slv5p" Sep 18 21:05:59.778135 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.778107 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-slv5p" Sep 18 21:05:59.824474 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.824446 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 18 21:05:59.839226 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839191 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-daemon-config\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.839226 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839232 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dqvx\" (UniqueName: \"kubernetes.io/projected/535650fe-fb2f-4b86-9c22-23eec74be1a1-kube-api-access-4dqvx\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.839488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839259 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvvfz\" (UniqueName: \"kubernetes.io/projected/133f1088-0c44-446d-832c-1d893f21ab04-kube-api-access-vvvfz\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.839488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839287 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:05:59.839488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cbba6619-7498-4ef7-a9a8-174ffc7c3953-kubelet-config\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:05:59.839488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839359 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pqzn\" (UniqueName: \"kubernetes.io/projected/20959b5d-e329-49fe-8e03-617b788df45f-kube-api-access-8pqzn\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.839488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839389 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-run-netns\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.839488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839415 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/133f1088-0c44-446d-832c-1d893f21ab04-etc-tuned\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.839488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839440 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/133f1088-0c44-446d-832c-1d893f21ab04-tmp\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.839488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839465 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839490 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/597032d6-55fc-4ec1-a791-533a72693ad2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.839498 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839515 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-node-log\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839537 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5f92b3a5-37b0-4d61-a166-a35c2b44d237-ovnkube-script-lib\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.839579 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs podName:3ff122f5-c7e3-4cc6-b2a7-46b532f84d50 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:00.339552365 +0000 UTC m=+2.988149963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs") pod "network-metrics-daemon-bvvml" (UID: "3ff122f5-c7e3-4cc6-b2a7-46b532f84d50") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839600 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839629 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-run-k8s-cni-cncf-io\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839669 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-kubernetes\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839694 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cbba6619-7498-4ef7-a9a8-174ffc7c3953-dbus\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/597032d6-55fc-4ec1-a791-533a72693ad2-system-cni-dir\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839744 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-systemd-units\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839767 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-etc-openvswitch\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839803 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-system-cni-dir\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839828 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-var-lib-kubelet\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839865 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-var-lib-kubelet\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.839894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839892 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839897 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839918 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/369e25ea-147c-4099-a089-03876f8f9ec5-agent-certs\") pod \"konnectivity-agent-jhzsp\" (UID: \"369e25ea-147c-4099-a089-03876f8f9ec5\") " pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839946 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbjh8\" (UniqueName: \"kubernetes.io/projected/626b4d06-d3b0-44fb-94b9-94f3131d6afc-kube-api-access-jbjh8\") pod \"node-resolver-pj68m\" (UID: \"626b4d06-d3b0-44fb-94b9-94f3131d6afc\") " pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839973 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85eb09c2-5e23-418c-a7fc-e4d4fa760156-host-slash\") pod \"iptables-alerter-6nct6\" (UID: \"85eb09c2-5e23-418c-a7fc-e4d4fa760156\") " pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.839999 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-cnibin\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840001 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cbba6619-7498-4ef7-a9a8-174ffc7c3953-kubelet-config\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840023 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/535650fe-fb2f-4b86-9c22-23eec74be1a1-cni-binary-copy\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840047 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28264d96-1e7c-49bf-9654-5ad9c675d9b6-host\") pod \"node-ca-pvrc4\" (UID: \"28264d96-1e7c-49bf-9654-5ad9c675d9b6\") " pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840070 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-sysctl-d\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840093 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/369e25ea-147c-4099-a089-03876f8f9ec5-konnectivity-ca\") pod \"konnectivity-agent-jhzsp\" (UID: \"369e25ea-147c-4099-a089-03876f8f9ec5\") " pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840123 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-kubernetes\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840173 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/597032d6-55fc-4ec1-a791-533a72693ad2-system-cni-dir\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.840216 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840216 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-var-lib-kubelet\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840251 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-socket-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.840267 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret podName:cbba6619-7498-4ef7-a9a8-174ffc7c3953 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:00.340249455 +0000 UTC m=+2.988847032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret") pod "global-pull-secret-syncer-hrxgz" (UID: "cbba6619-7498-4ef7-a9a8-174ffc7c3953") : object "kube-system"/"original-pull-secret" not registered Sep 18 21:05:59.840614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840311 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28264d96-1e7c-49bf-9654-5ad9c675d9b6-host\") pod \"node-ca-pvrc4\" (UID: \"28264d96-1e7c-49bf-9654-5ad9c675d9b6\") " pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840356 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840393 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-sysctl-d\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840437 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-etc-selinux\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840467 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/597032d6-55fc-4ec1-a791-533a72693ad2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840497 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-run-systemd\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840514 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-var-lib-cni-multus\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4bfd\" (UniqueName: \"kubernetes.io/projected/28264d96-1e7c-49bf-9654-5ad9c675d9b6-kube-api-access-x4bfd\") pod \"node-ca-pvrc4\" (UID: \"28264d96-1e7c-49bf-9654-5ad9c675d9b6\") " pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840612 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-device-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840699 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-slash\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840744 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-cni-bin\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840757 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-socket-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840776 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840805 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5f92b3a5-37b0-4d61-a166-a35c2b44d237-ovnkube-config\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840830 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85eb09c2-5e23-418c-a7fc-e4d4fa760156-iptables-alerter-script\") pod \"iptables-alerter-6nct6\" (UID: \"85eb09c2-5e23-418c-a7fc-e4d4fa760156\") " pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840854 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-cni-dir\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840877 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cbba6619-7498-4ef7-a9a8-174ffc7c3953-dbus\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:05:59.841215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840884 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/597032d6-55fc-4ec1-a791-533a72693ad2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840883 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/369e25ea-147c-4099-a089-03876f8f9ec5-konnectivity-ca\") pod \"konnectivity-agent-jhzsp\" (UID: \"369e25ea-147c-4099-a089-03876f8f9ec5\") " pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.840888 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-systemd\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841004 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-etc-selinux\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841021 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-run\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841051 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4r62\" (UniqueName: \"kubernetes.io/projected/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-kube-api-access-s4r62\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841060 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-device-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841079 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/597032d6-55fc-4ec1-a791-533a72693ad2-cni-binary-copy\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841110 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-cni-netd\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841138 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-socket-dir-parent\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841146 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-systemd\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841218 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/597032d6-55fc-4ec1-a791-533a72693ad2-cnibin\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftc5d\" (UniqueName: \"kubernetes.io/projected/85eb09c2-5e23-418c-a7fc-e4d4fa760156-kube-api-access-ftc5d\") pod \"iptables-alerter-6nct6\" (UID: \"85eb09c2-5e23-418c-a7fc-e4d4fa760156\") " pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841277 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-run-netns\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841300 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-run\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841302 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-var-lib-openvswitch\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841345 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-hostroot\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.842094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841376 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-etc-kubernetes\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841433 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-lib-modules\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/626b4d06-d3b0-44fb-94b9-94f3131d6afc-hosts-file\") pod \"node-resolver-pj68m\" (UID: \"626b4d06-d3b0-44fb-94b9-94f3131d6afc\") " pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841489 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-kubelet\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841514 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-run-multus-certs\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841566 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-log-socket\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841599 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-host\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841628 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlkhl\" (UniqueName: \"kubernetes.io/projected/597032d6-55fc-4ec1-a791-533a72693ad2-kube-api-access-zlkhl\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841666 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/597032d6-55fc-4ec1-a791-533a72693ad2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841701 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-lib-modules\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-run-openvswitch\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841792 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-var-lib-cni-bin\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841798 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/597032d6-55fc-4ec1-a791-533a72693ad2-cni-binary-copy\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841819 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-conf-dir\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841856 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-run-ovn\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841870 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/626b4d06-d3b0-44fb-94b9-94f3131d6afc-hosts-file\") pod \"node-resolver-pj68m\" (UID: \"626b4d06-d3b0-44fb-94b9-94f3131d6afc\") " pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841890 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-modprobe-d\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.842924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841921 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-registration-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841947 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/626b4d06-d3b0-44fb-94b9-94f3131d6afc-tmp-dir\") pod \"node-resolver-pj68m\" (UID: \"626b4d06-d3b0-44fb-94b9-94f3131d6afc\") " pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.841972 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-sys\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842000 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-run-ovn-kubernetes\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842055 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-host\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842113 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/597032d6-55fc-4ec1-a791-533a72693ad2-cnibin\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842225 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-modprobe-d\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842279 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-sys\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842316 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28264d96-1e7c-49bf-9654-5ad9c675d9b6-serviceca\") pod \"node-ca-pvrc4\" (UID: \"28264d96-1e7c-49bf-9654-5ad9c675d9b6\") " pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842348 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-registration-dir\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842366 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bdn8\" (UniqueName: \"kubernetes.io/projected/5f92b3a5-37b0-4d61-a166-a35c2b44d237-kube-api-access-5bdn8\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842439 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/626b4d06-d3b0-44fb-94b9-94f3131d6afc-tmp-dir\") pod \"node-resolver-pj68m\" (UID: \"626b4d06-d3b0-44fb-94b9-94f3131d6afc\") " pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842480 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-sysconfig\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842511 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-sysctl-conf\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842557 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/597032d6-55fc-4ec1-a791-533a72693ad2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842586 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-sys-fs\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842623 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-sysctl-conf\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.843846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842624 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5f92b3a5-37b0-4d61-a166-a35c2b44d237-env-overrides\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.844750 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842662 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f92b3a5-37b0-4d61-a166-a35c2b44d237-ovn-node-metrics-cert\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.844750 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842672 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/20959b5d-e329-49fe-8e03-617b788df45f-sys-fs\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.844750 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842758 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/133f1088-0c44-446d-832c-1d893f21ab04-etc-sysconfig\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.844750 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.842817 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/597032d6-55fc-4ec1-a791-533a72693ad2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.844750 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.843681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28264d96-1e7c-49bf-9654-5ad9c675d9b6-serviceca\") pod \"node-ca-pvrc4\" (UID: \"28264d96-1e7c-49bf-9654-5ad9c675d9b6\") " pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.845200 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.845180 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/133f1088-0c44-446d-832c-1d893f21ab04-etc-tuned\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.845268 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.845204 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/133f1088-0c44-446d-832c-1d893f21ab04-tmp\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.845433 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.845409 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/369e25ea-147c-4099-a089-03876f8f9ec5-agent-certs\") pod \"konnectivity-agent-jhzsp\" (UID: \"369e25ea-147c-4099-a089-03876f8f9ec5\") " pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:05:59.853552 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.853468 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvvfz\" (UniqueName: \"kubernetes.io/projected/133f1088-0c44-446d-832c-1d893f21ab04-kube-api-access-vvvfz\") pod \"tuned-9jqg2\" (UID: \"133f1088-0c44-446d-832c-1d893f21ab04\") " pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:05:59.853692 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.853640 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pqzn\" (UniqueName: \"kubernetes.io/projected/20959b5d-e329-49fe-8e03-617b788df45f-kube-api-access-8pqzn\") pod \"aws-ebs-csi-driver-node-94dnk\" (UID: \"20959b5d-e329-49fe-8e03-617b788df45f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:05:59.857006 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.856953 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4bfd\" (UniqueName: \"kubernetes.io/projected/28264d96-1e7c-49bf-9654-5ad9c675d9b6-kube-api-access-x4bfd\") pod \"node-ca-pvrc4\" (UID: \"28264d96-1e7c-49bf-9654-5ad9c675d9b6\") " pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:05:59.857201 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.857148 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4r62\" (UniqueName: \"kubernetes.io/projected/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-kube-api-access-s4r62\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:05:59.857894 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.857870 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlkhl\" (UniqueName: \"kubernetes.io/projected/597032d6-55fc-4ec1-a791-533a72693ad2-kube-api-access-zlkhl\") pod \"multus-additional-cni-plugins-9sm5k\" (UID: \"597032d6-55fc-4ec1-a791-533a72693ad2\") " pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:05:59.858339 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.858291 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbjh8\" (UniqueName: \"kubernetes.io/projected/626b4d06-d3b0-44fb-94b9-94f3131d6afc-kube-api-access-jbjh8\") pod \"node-resolver-pj68m\" (UID: \"626b4d06-d3b0-44fb-94b9-94f3131d6afc\") " pod="openshift-dns/node-resolver-pj68m" Sep 18 21:05:59.861462 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.861417 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal" event={"ID":"0177e4b7799939fec863354aec680a58","Type":"ContainerStarted","Data":"c6ddb433c3ef96a69e7f3c66848d48a2a95b391aae2173ee6bbb21aabeb39046"} Sep 18 21:05:59.863313 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.863258 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" event={"ID":"d0288c42e3cdc1f6a95ea48965eccccf","Type":"ContainerStarted","Data":"47e32636a6b91619a1057b870d423731b5febec45340967b21d89eb26ae02bbb"} Sep 18 21:05:59.943805 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.943772 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-cni-netd\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.943805 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.943808 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-socket-dir-parent\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.943825 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftc5d\" (UniqueName: \"kubernetes.io/projected/85eb09c2-5e23-418c-a7fc-e4d4fa760156-kube-api-access-ftc5d\") pod \"iptables-alerter-6nct6\" (UID: \"85eb09c2-5e23-418c-a7fc-e4d4fa760156\") " pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.943841 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-run-netns\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.943922 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-socket-dir-parent\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.943923 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-cni-netd\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.943960 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-var-lib-openvswitch\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.943994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-hostroot\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944000 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-run-netns\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944026 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-etc-kubernetes\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944040 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-hostroot\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944055 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-kubelet\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944080 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-etc-kubernetes\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-run-multus-certs\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944119 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-log-socket\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944116 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-run-multus-certs\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944123 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-kubelet\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944147 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-run-openvswitch\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944074 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-var-lib-openvswitch\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944170 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-var-lib-cni-bin\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944186 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-conf-dir\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944204 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-run-ovn\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944216 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-run-openvswitch\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944233 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-run-ovn-kubernetes\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944250 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-var-lib-cni-bin\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944262 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bdn8\" (UniqueName: \"kubernetes.io/projected/5f92b3a5-37b0-4d61-a166-a35c2b44d237-kube-api-access-5bdn8\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944255 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-log-socket\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944270 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-conf-dir\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944273 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-run-ovn\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944295 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5f92b3a5-37b0-4d61-a166-a35c2b44d237-env-overrides\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.944714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944298 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-run-ovn-kubernetes\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944321 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f92b3a5-37b0-4d61-a166-a35c2b44d237-ovn-node-metrics-cert\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944341 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-daemon-config\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944380 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dqvx\" (UniqueName: \"kubernetes.io/projected/535650fe-fb2f-4b86-9c22-23eec74be1a1-kube-api-access-4dqvx\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944634 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-run-netns\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944669 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-node-log\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944696 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5f92b3a5-37b0-4d61-a166-a35c2b44d237-ovnkube-script-lib\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944758 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-run-k8s-cni-cncf-io\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944787 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-systemd-units\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944811 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-etc-openvswitch\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944834 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-system-cni-dir\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944859 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-var-lib-kubelet\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944894 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5f92b3a5-37b0-4d61-a166-a35c2b44d237-env-overrides\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944900 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85eb09c2-5e23-418c-a7fc-e4d4fa760156-host-slash\") pod \"iptables-alerter-6nct6\" (UID: \"85eb09c2-5e23-418c-a7fc-e4d4fa760156\") " pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944945 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85eb09c2-5e23-418c-a7fc-e4d4fa760156-host-slash\") pod \"iptables-alerter-6nct6\" (UID: \"85eb09c2-5e23-418c-a7fc-e4d4fa760156\") " pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944948 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-cnibin\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.944978 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/535650fe-fb2f-4b86-9c22-23eec74be1a1-cni-binary-copy\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.945628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945022 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-daemon-config\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945029 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-var-lib-kubelet\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945075 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-etc-openvswitch\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945082 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-run-netns\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945112 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-run-k8s-cni-cncf-io\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945125 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-node-log\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945159 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-systemd-units\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945173 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-cnibin\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945200 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-run-systemd\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-var-lib-cni-multus\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945273 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-host-var-lib-cni-multus\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945296 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-system-cni-dir\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945314 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-run-systemd\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945353 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-slash\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945413 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-cni-bin\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945414 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5f92b3a5-37b0-4d61-a166-a35c2b44d237-ovnkube-script-lib\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945446 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-cni-bin\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945494 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.946577 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945543 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.947496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945548 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f92b3a5-37b0-4d61-a166-a35c2b44d237-host-slash\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.947496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945572 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5f92b3a5-37b0-4d61-a166-a35c2b44d237-ovnkube-config\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.947496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945613 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85eb09c2-5e23-418c-a7fc-e4d4fa760156-iptables-alerter-script\") pod \"iptables-alerter-6nct6\" (UID: \"85eb09c2-5e23-418c-a7fc-e4d4fa760156\") " pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.947496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945639 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-cni-dir\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.947496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945643 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/535650fe-fb2f-4b86-9c22-23eec74be1a1-cni-binary-copy\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.947496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.945790 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/535650fe-fb2f-4b86-9c22-23eec74be1a1-multus-cni-dir\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:05:59.947496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.946044 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5f92b3a5-37b0-4d61-a166-a35c2b44d237-ovnkube-config\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.947496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.946095 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85eb09c2-5e23-418c-a7fc-e4d4fa760156-iptables-alerter-script\") pod \"iptables-alerter-6nct6\" (UID: \"85eb09c2-5e23-418c-a7fc-e4d4fa760156\") " pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.947496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.947079 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f92b3a5-37b0-4d61-a166-a35c2b44d237-ovn-node-metrics-cert\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.956708 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.956679 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 21:05:59.956708 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.956702 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 21:05:59.956708 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.956712 2557 projected.go:196] Error preparing data for projected volume kube-api-access-dnvkl for pod openshift-network-diagnostics/network-check-target-6cmhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:05:59.956988 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:05:59.956789 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl podName:a565cb5c-0de6-4cf9-92de-a12b10b584ad nodeName:}" failed. No retries permitted until 2026-09-18 21:06:00.45676856 +0000 UTC m=+3.105366149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dnvkl" (UniqueName: "kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl") pod "network-check-target-6cmhj" (UID: "a565cb5c-0de6-4cf9-92de-a12b10b584ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:05:59.959195 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.959158 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftc5d\" (UniqueName: \"kubernetes.io/projected/85eb09c2-5e23-418c-a7fc-e4d4fa760156-kube-api-access-ftc5d\") pod \"iptables-alerter-6nct6\" (UID: \"85eb09c2-5e23-418c-a7fc-e4d4fa760156\") " pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:05:59.960496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.960473 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bdn8\" (UniqueName: \"kubernetes.io/projected/5f92b3a5-37b0-4d61-a166-a35c2b44d237-kube-api-access-5bdn8\") pod \"ovnkube-node-fqsfb\" (UID: \"5f92b3a5-37b0-4d61-a166-a35c2b44d237\") " pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:05:59.960659 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:05:59.960642 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dqvx\" (UniqueName: \"kubernetes.io/projected/535650fe-fb2f-4b86-9c22-23eec74be1a1-kube-api-access-4dqvx\") pod \"multus-2pc68\" (UID: \"535650fe-fb2f-4b86-9c22-23eec74be1a1\") " pod="openshift-multus/multus-2pc68" Sep 18 21:06:00.026371 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.026280 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pvrc4" Sep 18 21:06:00.035452 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.035424 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:06:00.045374 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.045349 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" Sep 18 21:06:00.054498 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.054474 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" Sep 18 21:06:00.064432 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.064406 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pj68m" Sep 18 21:06:00.072040 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.072020 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" Sep 18 21:06:00.082840 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.082810 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6nct6" Sep 18 21:06:00.091813 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.091791 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:06:00.098616 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.098597 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2pc68" Sep 18 21:06:00.350208 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.350103 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:00.350208 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.350195 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:00.350476 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:00.350289 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:00.350476 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:00.350310 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:00.350476 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:00.350398 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs podName:3ff122f5-c7e3-4cc6-b2a7-46b532f84d50 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:01.350377063 +0000 UTC m=+3.998974648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs") pod "network-metrics-daemon-bvvml" (UID: "3ff122f5-c7e3-4cc6-b2a7-46b532f84d50") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:00.350476 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:00.350415 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret podName:cbba6619-7498-4ef7-a9a8-174ffc7c3953 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:01.35040756 +0000 UTC m=+3.999005130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret") pod "global-pull-secret-syncer-hrxgz" (UID: "cbba6619-7498-4ef7-a9a8-174ffc7c3953") : object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:00.551556 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.551508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:00.551770 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:00.551733 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 21:06:00.551770 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:00.551763 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 21:06:00.551888 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:00.551779 2557 projected.go:196] Error preparing data for projected volume kube-api-access-dnvkl for pod openshift-network-diagnostics/network-check-target-6cmhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:00.551888 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:00.551856 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl podName:a565cb5c-0de6-4cf9-92de-a12b10b584ad nodeName:}" failed. No retries permitted until 2026-09-18 21:06:01.551835666 +0000 UTC m=+4.200433238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnvkl" (UniqueName: "kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl") pod "network-check-target-6cmhj" (UID: "a565cb5c-0de6-4cf9-92de-a12b10b584ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:00.780968 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.780871 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 21:01:00 +0000 UTC" deadline="2028-03-19 09:45:15.871733364 +0000 UTC" Sep 18 21:06:00.780968 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:00.780911 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13140h39m15.090825584s" Sep 18 21:06:00.947258 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:00.947232 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133f1088_0c44_446d_832c_1d893f21ab04.slice/crio-d7de3ac4c96cc2b22aeb26a45aeced260f346b5add5f0c0b28910cba9da37014 WatchSource:0}: Error finding container d7de3ac4c96cc2b22aeb26a45aeced260f346b5add5f0c0b28910cba9da37014: Status 404 returned error can't find the container with id d7de3ac4c96cc2b22aeb26a45aeced260f346b5add5f0c0b28910cba9da37014 Sep 18 21:06:00.951790 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:00.951758 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626b4d06_d3b0_44fb_94b9_94f3131d6afc.slice/crio-739b01fa1cfa6389423745c842720bbf46220613635401842daeca52739165ec WatchSource:0}: Error finding container 739b01fa1cfa6389423745c842720bbf46220613635401842daeca52739165ec: Status 404 returned error can't find the container with id 739b01fa1cfa6389423745c842720bbf46220613635401842daeca52739165ec Sep 18 21:06:00.954235 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:00.952783 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28264d96_1e7c_49bf_9654_5ad9c675d9b6.slice/crio-f75f54502f508d689454655e57c67bd85a9b64e8c25b81e83253b04b8bf0053b WatchSource:0}: Error finding container f75f54502f508d689454655e57c67bd85a9b64e8c25b81e83253b04b8bf0053b: Status 404 returned error can't find the container with id f75f54502f508d689454655e57c67bd85a9b64e8c25b81e83253b04b8bf0053b Sep 18 21:06:00.954235 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:00.953491 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85eb09c2_5e23_418c_a7fc_e4d4fa760156.slice/crio-51e9c9f2f8d597a800f3f27dd392527b3ae18e08de902901b832620bf3fcef0f WatchSource:0}: Error finding container 51e9c9f2f8d597a800f3f27dd392527b3ae18e08de902901b832620bf3fcef0f: Status 404 returned error can't find the container with id 51e9c9f2f8d597a800f3f27dd392527b3ae18e08de902901b832620bf3fcef0f Sep 18 21:06:00.957815 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:00.957780 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f92b3a5_37b0_4d61_a166_a35c2b44d237.slice/crio-87b3f9121fd2423b20836c6e873eabd46057df68544593ec99d164794a1c1da4 WatchSource:0}: Error finding container 87b3f9121fd2423b20836c6e873eabd46057df68544593ec99d164794a1c1da4: Status 404 returned error can't find the container with id 87b3f9121fd2423b20836c6e873eabd46057df68544593ec99d164794a1c1da4 Sep 18 21:06:00.958678 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:00.958643 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20959b5d_e329_49fe_8e03_617b788df45f.slice/crio-44e3d922e0c07aca58d7494306d0c7ef93b05e607459e37c55cc0c8c819eb0c0 WatchSource:0}: Error finding container 44e3d922e0c07aca58d7494306d0c7ef93b05e607459e37c55cc0c8c819eb0c0: Status 404 returned error can't find the container with id 44e3d922e0c07aca58d7494306d0c7ef93b05e607459e37c55cc0c8c819eb0c0 Sep 18 21:06:00.959784 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:00.959766 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597032d6_55fc_4ec1_a791_533a72693ad2.slice/crio-19eb63adabd9765e03375687f49c4b5c1c9454cc05d3f1f1e4de1a1b85f627f1 WatchSource:0}: Error finding container 19eb63adabd9765e03375687f49c4b5c1c9454cc05d3f1f1e4de1a1b85f627f1: Status 404 returned error can't find the container with id 19eb63adabd9765e03375687f49c4b5c1c9454cc05d3f1f1e4de1a1b85f627f1 Sep 18 21:06:00.961900 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:00.961804 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535650fe_fb2f_4b86_9c22_23eec74be1a1.slice/crio-f47ed769e59f501fdc0276a06a366dbfb6aeee6d7280b70e3b377c4fbe54252e WatchSource:0}: Error finding container f47ed769e59f501fdc0276a06a366dbfb6aeee6d7280b70e3b377c4fbe54252e: Status 404 returned error can't find the container with id f47ed769e59f501fdc0276a06a366dbfb6aeee6d7280b70e3b377c4fbe54252e Sep 18 21:06:00.962576 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:00.962552 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod369e25ea_147c_4099_a089_03876f8f9ec5.slice/crio-d49ca1b5ad79bd57e7e2de18aeaf927cf2a6170d7ba22a3058992459f991543f WatchSource:0}: Error finding container d49ca1b5ad79bd57e7e2de18aeaf927cf2a6170d7ba22a3058992459f991543f: Status 404 returned error can't find the container with id d49ca1b5ad79bd57e7e2de18aeaf927cf2a6170d7ba22a3058992459f991543f Sep 18 21:06:01.359668 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.359628 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:01.359872 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.359694 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:01.359872 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.359821 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:01.359973 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.359889 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret podName:cbba6619-7498-4ef7-a9a8-174ffc7c3953 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:03.359869725 +0000 UTC m=+6.008467309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret") pod "global-pull-secret-syncer-hrxgz" (UID: "cbba6619-7498-4ef7-a9a8-174ffc7c3953") : object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:01.359973 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.359818 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:01.360050 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.359981 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs podName:3ff122f5-c7e3-4cc6-b2a7-46b532f84d50 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:03.359960225 +0000 UTC m=+6.008557801 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs") pod "network-metrics-daemon-bvvml" (UID: "3ff122f5-c7e3-4cc6-b2a7-46b532f84d50") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:01.562375 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.562265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:01.562552 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.562461 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 21:06:01.562552 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.562489 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 21:06:01.562552 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.562512 2557 projected.go:196] Error preparing data for projected volume kube-api-access-dnvkl for pod openshift-network-diagnostics/network-check-target-6cmhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:01.562724 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.562581 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl podName:a565cb5c-0de6-4cf9-92de-a12b10b584ad nodeName:}" failed. No retries permitted until 2026-09-18 21:06:03.562558971 +0000 UTC m=+6.211156561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnvkl" (UniqueName: "kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl") pod "network-check-target-6cmhj" (UID: "a565cb5c-0de6-4cf9-92de-a12b10b584ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:01.781917 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.781864 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 21:01:00 +0000 UTC" deadline="2028-04-09 07:53:08.821954528 +0000 UTC" Sep 18 21:06:01.781917 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.781910 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13642h47m7.040049691s" Sep 18 21:06:01.863560 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.862043 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:01.863560 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.862189 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:01.863560 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.862617 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:01.863560 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.862719 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:01.863560 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.862799 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:01.863560 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:01.862920 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:01.899851 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.899772 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jhzsp" event={"ID":"369e25ea-147c-4099-a089-03876f8f9ec5","Type":"ContainerStarted","Data":"d49ca1b5ad79bd57e7e2de18aeaf927cf2a6170d7ba22a3058992459f991543f"} Sep 18 21:06:01.911482 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.911407 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" event={"ID":"20959b5d-e329-49fe-8e03-617b788df45f","Type":"ContainerStarted","Data":"44e3d922e0c07aca58d7494306d0c7ef93b05e607459e37c55cc0c8c819eb0c0"} Sep 18 21:06:01.913843 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.913807 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" event={"ID":"5f92b3a5-37b0-4d61-a166-a35c2b44d237","Type":"ContainerStarted","Data":"87b3f9121fd2423b20836c6e873eabd46057df68544593ec99d164794a1c1da4"} Sep 18 21:06:01.933384 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.933343 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" event={"ID":"133f1088-0c44-446d-832c-1d893f21ab04","Type":"ContainerStarted","Data":"d7de3ac4c96cc2b22aeb26a45aeced260f346b5add5f0c0b28910cba9da37014"} Sep 18 21:06:01.943650 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.943607 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2pc68" event={"ID":"535650fe-fb2f-4b86-9c22-23eec74be1a1","Type":"ContainerStarted","Data":"f47ed769e59f501fdc0276a06a366dbfb6aeee6d7280b70e3b377c4fbe54252e"} Sep 18 21:06:01.953584 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.953471 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6nct6" event={"ID":"85eb09c2-5e23-418c-a7fc-e4d4fa760156","Type":"ContainerStarted","Data":"51e9c9f2f8d597a800f3f27dd392527b3ae18e08de902901b832620bf3fcef0f"} Sep 18 21:06:01.960128 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.960089 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pvrc4" event={"ID":"28264d96-1e7c-49bf-9654-5ad9c675d9b6","Type":"ContainerStarted","Data":"f75f54502f508d689454655e57c67bd85a9b64e8c25b81e83253b04b8bf0053b"} Sep 18 21:06:01.969951 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.969876 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pj68m" event={"ID":"626b4d06-d3b0-44fb-94b9-94f3131d6afc","Type":"ContainerStarted","Data":"739b01fa1cfa6389423745c842720bbf46220613635401842daeca52739165ec"} Sep 18 21:06:01.985472 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.985340 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal" event={"ID":"0177e4b7799939fec863354aec680a58","Type":"ContainerStarted","Data":"03855fce4e36eed7bb97f8c6bb8da86303661a561263c4df32569c8b6c859e9e"} Sep 18 21:06:01.994015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.993216 2557 generic.go:356] "Generic (PLEG): container finished" podID="d0288c42e3cdc1f6a95ea48965eccccf" containerID="5d30693cf2967d83229537e70b650c88ea5c4ef3b94f6e0c295fff657292d26d" exitCode=0 Sep 18 21:06:01.994015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:01.993310 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" event={"ID":"d0288c42e3cdc1f6a95ea48965eccccf","Type":"ContainerDied","Data":"5d30693cf2967d83229537e70b650c88ea5c4ef3b94f6e0c295fff657292d26d"} Sep 18 21:06:02.006319 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:02.005737 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-163.ec2.internal" podStartSLOduration=3.00571508 podStartE2EDuration="3.00571508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:02.004688813 +0000 UTC m=+4.653286408" watchObservedRunningTime="2026-09-18 21:06:02.00571508 +0000 UTC m=+4.654312678" Sep 18 21:06:02.016291 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:02.016223 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" event={"ID":"597032d6-55fc-4ec1-a791-533a72693ad2","Type":"ContainerStarted","Data":"19eb63adabd9765e03375687f49c4b5c1c9454cc05d3f1f1e4de1a1b85f627f1"} Sep 18 21:06:03.051537 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:03.051089 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" event={"ID":"d0288c42e3cdc1f6a95ea48965eccccf","Type":"ContainerStarted","Data":"f5f53842d4d12d6915778e2baa20d56bab5b44e4487b58c21013bbcce6a7b481"} Sep 18 21:06:03.073563 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:03.073116 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-163.ec2.internal" podStartSLOduration=4.073091333 podStartE2EDuration="4.073091333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:03.073007788 +0000 UTC m=+5.721605383" watchObservedRunningTime="2026-09-18 21:06:03.073091333 +0000 UTC m=+5.721688928" Sep 18 21:06:03.388140 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:03.388035 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:03.388140 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:03.388120 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:03.388458 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.388301 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:03.388531 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.388441 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret podName:cbba6619-7498-4ef7-a9a8-174ffc7c3953 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:07.388371229 +0000 UTC m=+10.036968815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret") pod "global-pull-secret-syncer-hrxgz" (UID: "cbba6619-7498-4ef7-a9a8-174ffc7c3953") : object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:03.389163 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.389139 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:03.389271 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.389206 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs podName:3ff122f5-c7e3-4cc6-b2a7-46b532f84d50 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:07.389188785 +0000 UTC m=+10.037786359 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs") pod "network-metrics-daemon-bvvml" (UID: "3ff122f5-c7e3-4cc6-b2a7-46b532f84d50") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:03.589835 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:03.589792 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:03.590036 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.590009 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 21:06:03.590036 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.590034 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 21:06:03.590150 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.590048 2557 projected.go:196] Error preparing data for projected volume kube-api-access-dnvkl for pod openshift-network-diagnostics/network-check-target-6cmhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:03.590150 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.590116 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl podName:a565cb5c-0de6-4cf9-92de-a12b10b584ad nodeName:}" failed. No retries permitted until 2026-09-18 21:06:07.590095278 +0000 UTC m=+10.238692856 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnvkl" (UniqueName: "kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl") pod "network-check-target-6cmhj" (UID: "a565cb5c-0de6-4cf9-92de-a12b10b584ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:03.855728 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:03.855636 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:03.855906 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:03.855848 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:03.855906 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.855853 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:03.855906 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:03.855887 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:03.856073 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.856013 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:03.856178 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:03.856141 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:05.855552 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:05.855510 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:05.856036 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:05.855679 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:05.856036 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:05.855976 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:05.856169 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:05.856098 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:05.856223 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:05.856189 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:05.856466 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:05.856291 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:07.426964 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:07.426875 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:07.426964 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:07.426924 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:07.427501 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.427015 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:07.427501 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.427059 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret podName:cbba6619-7498-4ef7-a9a8-174ffc7c3953 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:15.427046408 +0000 UTC m=+18.075643976 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret") pod "global-pull-secret-syncer-hrxgz" (UID: "cbba6619-7498-4ef7-a9a8-174ffc7c3953") : object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:07.427501 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.427366 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:07.427501 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.427409 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs podName:3ff122f5-c7e3-4cc6-b2a7-46b532f84d50 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:15.427394084 +0000 UTC m=+18.075991655 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs") pod "network-metrics-daemon-bvvml" (UID: "3ff122f5-c7e3-4cc6-b2a7-46b532f84d50") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:07.628556 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:07.628513 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:07.628723 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.628710 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 21:06:07.628804 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.628731 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 21:06:07.628804 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.628744 2557 projected.go:196] Error preparing data for projected volume kube-api-access-dnvkl for pod openshift-network-diagnostics/network-check-target-6cmhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:07.628804 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.628802 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl podName:a565cb5c-0de6-4cf9-92de-a12b10b584ad nodeName:}" failed. No retries permitted until 2026-09-18 21:06:15.628783079 +0000 UTC m=+18.277380650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnvkl" (UniqueName: "kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl") pod "network-check-target-6cmhj" (UID: "a565cb5c-0de6-4cf9-92de-a12b10b584ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:07.855846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:07.855385 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:07.855846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:07.855500 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:07.855846 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.855513 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:07.855846 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.855626 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:07.855846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:07.855747 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:07.856200 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:07.855857 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:09.854182 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:09.854097 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:09.854182 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:09.854132 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:09.854710 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:09.854097 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:09.854710 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:09.854237 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:09.854710 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:09.854329 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:09.854710 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:09.854394 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:11.856607 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:11.856576 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:11.857018 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:11.856665 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:11.857018 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:11.856671 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:11.857018 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:11.856681 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:11.857018 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:11.856742 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:11.857018 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:11.856868 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:13.854579 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:13.854548 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:13.855058 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:13.854549 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:13.855058 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:13.854678 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:13.855058 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:13.854549 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:13.855058 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:13.854737 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:13.855058 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:13.854801 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:15.490155 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:15.490124 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:15.490568 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:15.490171 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:15.490568 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.490279 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:15.490568 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.490357 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs podName:3ff122f5-c7e3-4cc6-b2a7-46b532f84d50 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:31.490339758 +0000 UTC m=+34.138937326 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs") pod "network-metrics-daemon-bvvml" (UID: "3ff122f5-c7e3-4cc6-b2a7-46b532f84d50") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:15.490568 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.490277 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:15.490568 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.490425 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret podName:cbba6619-7498-4ef7-a9a8-174ffc7c3953 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:31.49041142 +0000 UTC m=+34.139008994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret") pod "global-pull-secret-syncer-hrxgz" (UID: "cbba6619-7498-4ef7-a9a8-174ffc7c3953") : object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:15.691492 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:15.691456 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:15.691685 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.691629 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 21:06:15.691685 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.691659 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 21:06:15.691685 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.691673 2557 projected.go:196] Error preparing data for projected volume kube-api-access-dnvkl for pod openshift-network-diagnostics/network-check-target-6cmhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:15.691836 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.691727 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl podName:a565cb5c-0de6-4cf9-92de-a12b10b584ad nodeName:}" failed. No retries permitted until 2026-09-18 21:06:31.691713283 +0000 UTC m=+34.340310852 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnvkl" (UniqueName: "kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl") pod "network-check-target-6cmhj" (UID: "a565cb5c-0de6-4cf9-92de-a12b10b584ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:15.854382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:15.854277 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:15.854382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:15.854301 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:15.854382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:15.854277 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:15.854621 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.854414 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:15.854621 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.854472 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:15.854621 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:15.854560 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:17.855709 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:17.855668 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:17.856186 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:17.855721 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:17.856186 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:17.855811 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:17.856186 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:17.855880 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:17.856186 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:17.855945 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:17.856186 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:17.856023 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:19.854686 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:19.854520 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:19.855440 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:19.854520 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:19.855440 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:19.854767 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:19.855440 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:19.854821 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:19.855440 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:19.854578 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:19.855440 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:19.854887 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:20.094844 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.094762 2557 generic.go:356] "Generic (PLEG): container finished" podID="597032d6-55fc-4ec1-a791-533a72693ad2" containerID="84f1594ec23b5ba8e197070c6e3fe237e4d7c33838f8fbb00d3126285969599d" exitCode=0 Sep 18 21:06:20.094844 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.094801 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" event={"ID":"597032d6-55fc-4ec1-a791-533a72693ad2","Type":"ContainerDied","Data":"84f1594ec23b5ba8e197070c6e3fe237e4d7c33838f8fbb00d3126285969599d"} Sep 18 21:06:20.096832 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.096803 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jhzsp" event={"ID":"369e25ea-147c-4099-a089-03876f8f9ec5","Type":"ContainerStarted","Data":"763ac4c04d4073a412a2126ad29a99784671b03658e06fceebead9eba302d460"} Sep 18 21:06:20.098426 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.098403 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" event={"ID":"20959b5d-e329-49fe-8e03-617b788df45f","Type":"ContainerStarted","Data":"b8d93b4e7180c4ae519c759e44c9a0beaf5660e27657e64ad6fe5e776f2264cc"} Sep 18 21:06:20.102175 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.102150 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" event={"ID":"5f92b3a5-37b0-4d61-a166-a35c2b44d237","Type":"ContainerStarted","Data":"0a7a4c064c8efe0514e47b46960a15633d99941a1645627111ce237ceb92d154"} Sep 18 21:06:20.102272 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.102231 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" event={"ID":"5f92b3a5-37b0-4d61-a166-a35c2b44d237","Type":"ContainerStarted","Data":"cb6898700585da6b213c9d2da5129a3ea382235648dd37fc19e0f2ca21a67e1a"} Sep 18 21:06:20.102344 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.102280 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" event={"ID":"5f92b3a5-37b0-4d61-a166-a35c2b44d237","Type":"ContainerStarted","Data":"86872ab2e70fa9e96253190f300556acad501b50ffa833ba1554538882c5dfc1"} Sep 18 21:06:20.102344 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.102340 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" event={"ID":"5f92b3a5-37b0-4d61-a166-a35c2b44d237","Type":"ContainerStarted","Data":"0a1ea1b0a566f51bd7f83fbb9f09c0c7e78cc1b59679ba8863db8f1773b63a1f"} Sep 18 21:06:20.102443 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.102382 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" event={"ID":"5f92b3a5-37b0-4d61-a166-a35c2b44d237","Type":"ContainerStarted","Data":"9740a708bbb30201a6a644ac503831c4be11738ed2a382f1a8e1fe0d49f55968"} Sep 18 21:06:20.102443 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.102423 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" event={"ID":"5f92b3a5-37b0-4d61-a166-a35c2b44d237","Type":"ContainerStarted","Data":"7c75f097b44076368251bdff5028c8257af660ccc357a6d2b0ad56a2bcc41448"} Sep 18 21:06:20.103873 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.103852 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" event={"ID":"133f1088-0c44-446d-832c-1d893f21ab04","Type":"ContainerStarted","Data":"bb8b80ae43607f50f3e44a52b49df80e7287ad8813ecd80a19d0edc670fc040b"} Sep 18 21:06:20.105599 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.105577 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2pc68" event={"ID":"535650fe-fb2f-4b86-9c22-23eec74be1a1","Type":"ContainerStarted","Data":"a1a823f3b2379156c7226c725690a3f09bf091568a32c070e152989bca48e0b0"} Sep 18 21:06:20.107255 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.107214 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pvrc4" event={"ID":"28264d96-1e7c-49bf-9654-5ad9c675d9b6","Type":"ContainerStarted","Data":"176c3e26701346091e5f54eec196183a37e8645d883774e856d7590866496624"} Sep 18 21:06:20.108879 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.108857 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pj68m" event={"ID":"626b4d06-d3b0-44fb-94b9-94f3131d6afc","Type":"ContainerStarted","Data":"99f877d70f5f1b1bd3bf3da77d718e0884fed732f932a9844a32f7e8c9c32118"} Sep 18 21:06:20.151511 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.151450 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9jqg2" podStartSLOduration=3.910226277 podStartE2EDuration="22.151432356s" totalImagesPullingTime="18.241206079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:20.15142217 +0000 UTC m=+22.800019773" watchObservedRunningTime="2026-09-18 21:06:20.151432356 +0000 UTC m=+22.800029947" Sep 18 21:06:20.169499 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.169442 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pj68m" podStartSLOduration=3.939221642 podStartE2EDuration="22.169420927s" totalImagesPullingTime="18.230199285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:20.169043533 +0000 UTC m=+22.817641125" watchObservedRunningTime="2026-09-18 21:06:20.169420927 +0000 UTC m=+22.818018519" Sep 18 21:06:20.188203 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.188146 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jhzsp" podStartSLOduration=4.008577804 podStartE2EDuration="22.188129877s" totalImagesPullingTime="18.179552073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:20.187964594 +0000 UTC m=+22.836562186" watchObservedRunningTime="2026-09-18 21:06:20.188129877 +0000 UTC m=+22.836727473" Sep 18 21:06:20.220996 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.220939 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2pc68" podStartSLOduration=3.947350367 podStartE2EDuration="22.220920359s" totalImagesPullingTime="18.273569992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:20.220533212 +0000 UTC m=+22.869130803" watchObservedRunningTime="2026-09-18 21:06:20.220920359 +0000 UTC m=+22.869517953" Sep 18 21:06:20.914945 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:20.914914 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 18 21:06:21.114361 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:21.114241 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" event={"ID":"20959b5d-e329-49fe-8e03-617b788df45f","Type":"ContainerStarted","Data":"b833c0c9d0b16dbe364c6a27bbf25328a55e13c8deb780d912c9a68987d33da7"} Sep 18 21:06:21.115949 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:21.115915 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6nct6" event={"ID":"85eb09c2-5e23-418c-a7fc-e4d4fa760156","Type":"ContainerStarted","Data":"0ca9dd1b57fbe9783560fe6239e185b99b775fbe9ae3b9b3e9b54041a9eb27e8"} Sep 18 21:06:21.140700 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:21.140645 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pvrc4" podStartSLOduration=4.923285366 podStartE2EDuration="23.140627426s" totalImagesPullingTime="18.21734206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:20.247005547 +0000 UTC m=+22.895603139" watchObservedRunningTime="2026-09-18 21:06:21.140627426 +0000 UTC m=+23.789225075" Sep 18 21:06:21.797412 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:21.797282 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T21:06:20.91493747Z","UUID":"e8d9d3f4-5e3f-417b-9c48-9ab4b61431a0","Handler":null,"Name":"","Endpoint":""} Sep 18 21:06:21.799187 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:21.799165 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 18 21:06:21.799356 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:21.799203 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 18 21:06:21.854475 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:21.854446 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:21.854641 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:21.854560 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:21.854641 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:21.854568 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:21.854744 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:21.854653 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:21.854744 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:21.854446 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:21.854830 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:21.854761 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:22.120884 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:22.120627 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" event={"ID":"20959b5d-e329-49fe-8e03-617b788df45f","Type":"ContainerStarted","Data":"514200ddd4bfbec70125380bf56e84befde5ad913769f738adabc522cd810ac1"} Sep 18 21:06:22.124367 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:22.124338 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" event={"ID":"5f92b3a5-37b0-4d61-a166-a35c2b44d237","Type":"ContainerStarted","Data":"33e4b43cc3b0ddba7f6889611d093e05a0e87f741a152fe766947fbdc77b81f5"} Sep 18 21:06:22.157662 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:22.157608 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6nct6" podStartSLOduration=5.939060102 podStartE2EDuration="24.157592054s" totalImagesPullingTime="18.218531952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:21.141978865 +0000 UTC m=+23.790576455" watchObservedRunningTime="2026-09-18 21:06:22.157592054 +0000 UTC m=+24.806189645" Sep 18 21:06:22.736806 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:22.736774 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:06:22.737587 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:22.737561 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:06:22.796185 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:22.796136 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94dnk" podStartSLOduration=4.19593332 podStartE2EDuration="24.796120862s" totalImagesPullingTime="20.600187542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:22.16258289 +0000 UTC m=+24.811180482" watchObservedRunningTime="2026-09-18 21:06:22.796120862 +0000 UTC m=+25.444718452" Sep 18 21:06:23.527819 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:23.527782 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:06:23.528422 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:23.528385 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jhzsp" Sep 18 21:06:23.854967 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:23.854885 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:23.855130 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:23.854885 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:23.855130 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:23.855044 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:23.855130 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:23.855094 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:23.855293 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:23.855134 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:23.855293 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:23.855216 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:25.134023 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.133768 2557 generic.go:356] "Generic (PLEG): container finished" podID="597032d6-55fc-4ec1-a791-533a72693ad2" containerID="9f341bc61095148283c79d04e813f585e059006be16dcf326e383fab54d8aa57" exitCode=0 Sep 18 21:06:25.134023 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.133858 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" event={"ID":"597032d6-55fc-4ec1-a791-533a72693ad2","Type":"ContainerDied","Data":"9f341bc61095148283c79d04e813f585e059006be16dcf326e383fab54d8aa57"} Sep 18 21:06:25.143272 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.143246 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" event={"ID":"5f92b3a5-37b0-4d61-a166-a35c2b44d237","Type":"ContainerStarted","Data":"5881b4ee25c01e92584abf10a9df219bca2e94bf57933c9196b99181be600566"} Sep 18 21:06:25.143508 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.143487 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:06:25.143588 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.143560 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:06:25.158754 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.158732 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:06:25.290483 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.290427 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" podStartSLOduration=9.016167806 podStartE2EDuration="27.290409511s" totalImagesPullingTime="18.274241705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:25.289401069 +0000 UTC m=+27.937998660" watchObservedRunningTime="2026-09-18 21:06:25.290409511 +0000 UTC m=+27.939007388" Sep 18 21:06:25.854002 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.853971 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:25.854151 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.853976 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:25.854151 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:25.854086 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:25.854236 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:25.853976 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:25.854236 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:25.854172 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:25.854308 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:25.854248 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:26.146247 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:26.146165 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:06:26.160831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:26.160807 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:06:27.037495 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:27.037290 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bvvml"] Sep 18 21:06:27.037678 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:27.037636 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:27.037772 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:27.037742 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:27.043093 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:27.043070 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6cmhj"] Sep 18 21:06:27.043191 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:27.043160 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:27.043241 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:27.043226 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:27.055804 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:27.055779 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hrxgz"] Sep 18 21:06:27.055924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:27.055894 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:27.056031 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:27.055993 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:27.149851 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:27.149817 2557 generic.go:356] "Generic (PLEG): container finished" podID="597032d6-55fc-4ec1-a791-533a72693ad2" containerID="ae5c289d09333b956da9c833fa72f1d0ed019989f1cba0b81bac15aad1a14297" exitCode=0 Sep 18 21:06:27.150217 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:27.149903 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" event={"ID":"597032d6-55fc-4ec1-a791-533a72693ad2","Type":"ContainerDied","Data":"ae5c289d09333b956da9c833fa72f1d0ed019989f1cba0b81bac15aad1a14297"} Sep 18 21:06:28.154673 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:28.154634 2557 generic.go:356] "Generic (PLEG): container finished" podID="597032d6-55fc-4ec1-a791-533a72693ad2" containerID="1cf5f10548e60a6950417019a95c6e611b7d0de82755d5d89e0f9ee780e4636f" exitCode=0 Sep 18 21:06:28.155076 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:28.154697 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" event={"ID":"597032d6-55fc-4ec1-a791-533a72693ad2","Type":"ContainerDied","Data":"1cf5f10548e60a6950417019a95c6e611b7d0de82755d5d89e0f9ee780e4636f"} Sep 18 21:06:28.854244 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:28.854158 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:28.854436 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:28.854158 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:28.854436 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:28.854284 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:28.854436 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:28.854158 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:28.854436 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:28.854380 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:28.854436 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:28.854427 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:30.854435 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:30.854400 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:30.854993 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:30.854401 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:30.854993 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:30.854536 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvvml" podUID="3ff122f5-c7e3-4cc6-b2a7-46b532f84d50" Sep 18 21:06:30.854993 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:30.854401 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:30.854993 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:30.854635 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cmhj" podUID="a565cb5c-0de6-4cf9-92de-a12b10b584ad" Sep 18 21:06:30.854993 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:30.854668 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrxgz" podUID="cbba6619-7498-4ef7-a9a8-174ffc7c3953" Sep 18 21:06:31.504505 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:31.504461 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:31.504703 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:31.504544 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:31.504703 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:31.504598 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:31.504703 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:31.504644 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:31.504703 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:31.504681 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret podName:cbba6619-7498-4ef7-a9a8-174ffc7c3953 nodeName:}" failed. No retries permitted until 2026-09-18 21:07:03.504660585 +0000 UTC m=+66.153258160 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret") pod "global-pull-secret-syncer-hrxgz" (UID: "cbba6619-7498-4ef7-a9a8-174ffc7c3953") : object "kube-system"/"original-pull-secret" not registered Sep 18 21:06:31.504703 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:31.504701 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs podName:3ff122f5-c7e3-4cc6-b2a7-46b532f84d50 nodeName:}" failed. No retries permitted until 2026-09-18 21:07:03.504693126 +0000 UTC m=+66.153290698 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs") pod "network-metrics-daemon-bvvml" (UID: "3ff122f5-c7e3-4cc6-b2a7-46b532f84d50") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 21:06:31.706692 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:31.706661 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:31.706858 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:31.706787 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 21:06:31.706858 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:31.706806 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 21:06:31.706858 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:31.706817 2557 projected.go:196] Error preparing data for projected volume kube-api-access-dnvkl for pod openshift-network-diagnostics/network-check-target-6cmhj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:31.706998 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:31.706882 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl podName:a565cb5c-0de6-4cf9-92de-a12b10b584ad nodeName:}" failed. No retries permitted until 2026-09-18 21:07:03.706865126 +0000 UTC m=+66.355462700 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnvkl" (UniqueName: "kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl") pod "network-check-target-6cmhj" (UID: "a565cb5c-0de6-4cf9-92de-a12b10b584ad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 21:06:32.165469 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.165438 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-163.ec2.internal" event="NodeReady" Sep 18 21:06:32.165871 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.165558 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 18 21:06:32.261012 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.260973 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b86549d88-bbwrm"] Sep 18 21:06:32.282304 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.282220 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.297150 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.297124 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 18 21:06:32.297293 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.297124 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-26f2c\"" Sep 18 21:06:32.297293 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.297123 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 18 21:06:32.297293 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.297173 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 18 21:06:32.303830 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.303811 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 18 21:06:32.311401 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.311374 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b86549d88-bbwrm"] Sep 18 21:06:32.314094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.314070 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5n7pb"] Sep 18 21:06:32.328598 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.328574 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5n7pb" Sep 18 21:06:32.333304 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.333285 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 18 21:06:32.333925 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.333905 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 18 21:06:32.334035 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.333935 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 18 21:06:32.334164 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.334151 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5tlbx\"" Sep 18 21:06:32.347286 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.347267 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5n7pb"] Sep 18 21:06:32.347990 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.347972 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tsxlb"] Sep 18 21:06:32.362700 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.362680 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.370323 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.370295 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 18 21:06:32.370323 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.370306 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 18 21:06:32.370465 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.370442 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qsk7s\"" Sep 18 21:06:32.399503 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.399461 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tsxlb"] Sep 18 21:06:32.412626 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412599 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-image-registry-private-configuration\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.412761 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412648 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-ca-trust-extracted\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.412761 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412733 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-certificates\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.412886 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.412886 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412782 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-trusted-ca\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.412886 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412799 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37a4721d-1c7f-434e-97d0-f1d32baae843-cert\") pod \"ingress-canary-5n7pb\" (UID: \"37a4721d-1c7f-434e-97d0-f1d32baae843\") " pod="openshift-ingress-canary/ingress-canary-5n7pb" Sep 18 21:06:32.412886 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412835 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qprmn\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-kube-api-access-qprmn\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.412886 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412869 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-installation-pull-secrets\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.413116 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412890 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-bound-sa-token\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.413116 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.412913 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22mjg\" (UniqueName: \"kubernetes.io/projected/37a4721d-1c7f-434e-97d0-f1d32baae843-kube-api-access-22mjg\") pod \"ingress-canary-5n7pb\" (UID: \"37a4721d-1c7f-434e-97d0-f1d32baae843\") " pod="openshift-ingress-canary/ingress-canary-5n7pb" Sep 18 21:06:32.513491 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-certificates\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.513664 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513494 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2321d814-a682-4e63-bc8b-c174d6cea7bc-config-volume\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.513664 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513528 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.513777 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513660 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-trusted-ca\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.513777 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513701 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37a4721d-1c7f-434e-97d0-f1d32baae843-cert\") pod \"ingress-canary-5n7pb\" (UID: \"37a4721d-1c7f-434e-97d0-f1d32baae843\") " pod="openshift-ingress-canary/ingress-canary-5n7pb" Sep 18 21:06:32.513777 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513733 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.513777 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:32.513670 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 21:06:32.513777 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:32.513771 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b86549d88-bbwrm: secret "image-registry-tls" not found Sep 18 21:06:32.514015 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:32.513833 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls podName:4251e8c2-55f8-488c-96f1-89cb5ffc93f0 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:33.013812204 +0000 UTC m=+35.662409777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls") pod "image-registry-6b86549d88-bbwrm" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0") : secret "image-registry-tls" not found Sep 18 21:06:32.514015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513863 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qprmn\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-kube-api-access-qprmn\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.514015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513900 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-installation-pull-secrets\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.514015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513918 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-bound-sa-token\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.514015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513940 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22mjg\" (UniqueName: \"kubernetes.io/projected/37a4721d-1c7f-434e-97d0-f1d32baae843-kube-api-access-22mjg\") pod \"ingress-canary-5n7pb\" (UID: \"37a4721d-1c7f-434e-97d0-f1d32baae843\") " pod="openshift-ingress-canary/ingress-canary-5n7pb" Sep 18 21:06:32.514015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.513971 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2321d814-a682-4e63-bc8b-c174d6cea7bc-tmp-dir\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.514015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.514012 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-image-registry-private-configuration\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.514400 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.514039 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvcst\" (UniqueName: \"kubernetes.io/projected/2321d814-a682-4e63-bc8b-c174d6cea7bc-kube-api-access-vvcst\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.514400 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.514089 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-ca-trust-extracted\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.514400 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.514154 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-certificates\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.514560 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.514534 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-trusted-ca\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.514600 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.514534 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-ca-trust-extracted\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.517757 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.517727 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-installation-pull-secrets\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.517757 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.517739 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-image-registry-private-configuration\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.517948 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.517912 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37a4721d-1c7f-434e-97d0-f1d32baae843-cert\") pod \"ingress-canary-5n7pb\" (UID: \"37a4721d-1c7f-434e-97d0-f1d32baae843\") " pod="openshift-ingress-canary/ingress-canary-5n7pb" Sep 18 21:06:32.535899 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.535833 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22mjg\" (UniqueName: \"kubernetes.io/projected/37a4721d-1c7f-434e-97d0-f1d32baae843-kube-api-access-22mjg\") pod \"ingress-canary-5n7pb\" (UID: \"37a4721d-1c7f-434e-97d0-f1d32baae843\") " pod="openshift-ingress-canary/ingress-canary-5n7pb" Sep 18 21:06:32.538739 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.538710 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-bound-sa-token\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.546633 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.546606 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qprmn\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-kube-api-access-qprmn\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:32.615176 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.615148 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2321d814-a682-4e63-bc8b-c174d6cea7bc-config-volume\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.615356 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.615189 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.615356 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.615219 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2321d814-a682-4e63-bc8b-c174d6cea7bc-tmp-dir\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.615356 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.615245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvcst\" (UniqueName: \"kubernetes.io/projected/2321d814-a682-4e63-bc8b-c174d6cea7bc-kube-api-access-vvcst\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.615528 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:32.615359 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 21:06:32.615528 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:32.615431 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls podName:2321d814-a682-4e63-bc8b-c174d6cea7bc nodeName:}" failed. No retries permitted until 2026-09-18 21:06:33.115408649 +0000 UTC m=+35.764006238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls") pod "dns-default-tsxlb" (UID: "2321d814-a682-4e63-bc8b-c174d6cea7bc") : secret "dns-default-metrics-tls" not found Sep 18 21:06:32.615666 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.615644 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2321d814-a682-4e63-bc8b-c174d6cea7bc-tmp-dir\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.615850 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.615828 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2321d814-a682-4e63-bc8b-c174d6cea7bc-config-volume\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.631330 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.631298 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvcst\" (UniqueName: \"kubernetes.io/projected/2321d814-a682-4e63-bc8b-c174d6cea7bc-kube-api-access-vvcst\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:32.637243 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.637222 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5n7pb" Sep 18 21:06:32.757189 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.757158 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jhzsp_369e25ea-147c-4099-a089-03876f8f9ec5/konnectivity-agent/0.log" limit=1024 Sep 18 21:06:32.854040 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.854010 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:06:32.854188 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.854049 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:06:32.854188 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.854118 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:06:32.857480 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.857454 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 21:06:32.858133 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.858116 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 21:06:32.858300 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.858229 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 21:06:32.858300 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.858295 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pzg28\"" Sep 18 21:06:32.858447 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.858298 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 21:06:32.858447 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.858235 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7qn9\"" Sep 18 21:06:32.884434 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.884404 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jhzsp_369e25ea-147c-4099-a089-03876f8f9ec5/konnectivity-agent/0.log" limit=1024 Sep 18 21:06:32.961638 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:32.961605 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5n7pb"] Sep 18 21:06:32.966041 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:06:32.966016 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a4721d_1c7f_434e_97d0_f1d32baae843.slice/crio-0c9b49cd83594432fe31a464e0cad4e39e409621c0fef7a3756866693099b095 WatchSource:0}: Error finding container 0c9b49cd83594432fe31a464e0cad4e39e409621c0fef7a3756866693099b095: Status 404 returned error can't find the container with id 0c9b49cd83594432fe31a464e0cad4e39e409621c0fef7a3756866693099b095 Sep 18 21:06:33.018471 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:33.018438 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:33.018639 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:33.018574 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 21:06:33.018639 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:33.018591 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b86549d88-bbwrm: secret "image-registry-tls" not found Sep 18 21:06:33.018715 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:33.018702 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls podName:4251e8c2-55f8-488c-96f1-89cb5ffc93f0 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:34.018682369 +0000 UTC m=+36.667279939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls") pod "image-registry-6b86549d88-bbwrm" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0") : secret "image-registry-tls" not found Sep 18 21:06:33.119798 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:33.119701 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:33.119950 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:33.119846 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 21:06:33.119950 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:33.119924 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls podName:2321d814-a682-4e63-bc8b-c174d6cea7bc nodeName:}" failed. No retries permitted until 2026-09-18 21:06:34.119908637 +0000 UTC m=+36.768506205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls") pod "dns-default-tsxlb" (UID: "2321d814-a682-4e63-bc8b-c174d6cea7bc") : secret "dns-default-metrics-tls" not found Sep 18 21:06:33.169737 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:33.169698 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5n7pb" event={"ID":"37a4721d-1c7f-434e-97d0-f1d32baae843","Type":"ContainerStarted","Data":"0c9b49cd83594432fe31a464e0cad4e39e409621c0fef7a3756866693099b095"} Sep 18 21:06:33.172466 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:33.172436 2557 generic.go:356] "Generic (PLEG): container finished" podID="597032d6-55fc-4ec1-a791-533a72693ad2" containerID="341f85a870e57ee758af988e979627727c95ae91fbc3f9bb9ca72d097bb8e79b" exitCode=0 Sep 18 21:06:33.172580 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:33.172486 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" event={"ID":"597032d6-55fc-4ec1-a791-533a72693ad2","Type":"ContainerDied","Data":"341f85a870e57ee758af988e979627727c95ae91fbc3f9bb9ca72d097bb8e79b"} Sep 18 21:06:34.028600 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:34.028563 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:34.028778 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:34.028706 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 21:06:34.028778 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:34.028719 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b86549d88-bbwrm: secret "image-registry-tls" not found Sep 18 21:06:34.028778 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:34.028773 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls podName:4251e8c2-55f8-488c-96f1-89cb5ffc93f0 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:36.028759349 +0000 UTC m=+38.677356918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls") pod "image-registry-6b86549d88-bbwrm" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0") : secret "image-registry-tls" not found Sep 18 21:06:34.129431 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:34.129388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:34.129690 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:34.129523 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 21:06:34.129690 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:34.129581 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls podName:2321d814-a682-4e63-bc8b-c174d6cea7bc nodeName:}" failed. No retries permitted until 2026-09-18 21:06:36.129566428 +0000 UTC m=+38.778164001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls") pod "dns-default-tsxlb" (UID: "2321d814-a682-4e63-bc8b-c174d6cea7bc") : secret "dns-default-metrics-tls" not found Sep 18 21:06:34.179206 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:34.179171 2557 generic.go:356] "Generic (PLEG): container finished" podID="597032d6-55fc-4ec1-a791-533a72693ad2" containerID="56ddd52036a7cb7aa69d867fb75c0a94ab6a685f82d92aced1b799c7bd693b92" exitCode=0 Sep 18 21:06:34.179660 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:34.179247 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" event={"ID":"597032d6-55fc-4ec1-a791-533a72693ad2","Type":"ContainerDied","Data":"56ddd52036a7cb7aa69d867fb75c0a94ab6a685f82d92aced1b799c7bd693b92"} Sep 18 21:06:35.186899 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:35.186723 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" event={"ID":"597032d6-55fc-4ec1-a791-533a72693ad2","Type":"ContainerStarted","Data":"577e3ef00924c116fa9d1b8f8bb05fa8a04e817f379d9f37447713ba020e9dd8"} Sep 18 21:06:35.218005 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:35.216611 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9sm5k" podStartSLOduration=8.710125521 podStartE2EDuration="37.216590443s" totalImagesPullingTime="28.506464922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:35.215234751 +0000 UTC m=+37.863832338" watchObservedRunningTime="2026-09-18 21:06:35.216590443 +0000 UTC m=+37.865188036" Sep 18 21:06:35.277235 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:35.277204 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jhzsp_369e25ea-147c-4099-a089-03876f8f9ec5/konnectivity-agent/0.log" limit=1024 Sep 18 21:06:35.355951 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:35.355906 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jhzsp_369e25ea-147c-4099-a089-03876f8f9ec5/konnectivity-agent/0.log" limit=1024 Sep 18 21:06:35.438080 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:35.438000 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jhzsp_369e25ea-147c-4099-a089-03876f8f9ec5/konnectivity-agent/0.log" limit=1024 Sep 18 21:06:36.046335 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:36.046276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:36.046495 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:36.046427 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 21:06:36.046495 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:36.046449 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b86549d88-bbwrm: secret "image-registry-tls" not found Sep 18 21:06:36.046558 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:36.046507 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls podName:4251e8c2-55f8-488c-96f1-89cb5ffc93f0 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:40.04649077 +0000 UTC m=+42.695088339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls") pod "image-registry-6b86549d88-bbwrm" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0") : secret "image-registry-tls" not found Sep 18 21:06:36.147015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:36.146978 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:36.147196 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:36.147107 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 21:06:36.147196 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:36.147168 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls podName:2321d814-a682-4e63-bc8b-c174d6cea7bc nodeName:}" failed. No retries permitted until 2026-09-18 21:06:40.14715204 +0000 UTC m=+42.795749612 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls") pod "dns-default-tsxlb" (UID: "2321d814-a682-4e63-bc8b-c174d6cea7bc") : secret "dns-default-metrics-tls" not found Sep 18 21:06:36.190375 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:36.190333 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5n7pb" event={"ID":"37a4721d-1c7f-434e-97d0-f1d32baae843","Type":"ContainerStarted","Data":"f37178ed1fe0c314f3c45a1f3ba7042ae5b8f5ca98b4f2a85ebe8af3deaf0dae"} Sep 18 21:06:36.208696 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:36.208341 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5n7pb" podStartSLOduration=1.5117320680000002 podStartE2EDuration="4.208302399s" totalImagesPullingTime="2.696570331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:06:36.207868639 +0000 UTC m=+38.856466241" watchObservedRunningTime="2026-09-18 21:06:36.208302399 +0000 UTC m=+38.856899992" Sep 18 21:06:37.382084 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:37.382042 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jhzsp_369e25ea-147c-4099-a089-03876f8f9ec5/konnectivity-agent/0.log" limit=1024 Sep 18 21:06:40.076930 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:40.076891 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:40.077286 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:40.077066 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 21:06:40.077286 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:40.077080 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b86549d88-bbwrm: secret "image-registry-tls" not found Sep 18 21:06:40.077286 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:40.077140 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls podName:4251e8c2-55f8-488c-96f1-89cb5ffc93f0 nodeName:}" failed. No retries permitted until 2026-09-18 21:06:48.077121942 +0000 UTC m=+50.725719521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls") pod "image-registry-6b86549d88-bbwrm" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0") : secret "image-registry-tls" not found Sep 18 21:06:40.178028 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:40.177993 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:40.178234 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:40.178152 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 21:06:40.178234 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:40.178214 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls podName:2321d814-a682-4e63-bc8b-c174d6cea7bc nodeName:}" failed. No retries permitted until 2026-09-18 21:06:48.178196405 +0000 UTC m=+50.826793975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls") pod "dns-default-tsxlb" (UID: "2321d814-a682-4e63-bc8b-c174d6cea7bc") : secret "dns-default-metrics-tls" not found Sep 18 21:06:48.135086 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:48.135044 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:06:48.135590 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:48.135173 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 21:06:48.135590 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:48.135185 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b86549d88-bbwrm: secret "image-registry-tls" not found Sep 18 21:06:48.135590 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:48.135247 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls podName:4251e8c2-55f8-488c-96f1-89cb5ffc93f0 nodeName:}" failed. No retries permitted until 2026-09-18 21:07:04.135232857 +0000 UTC m=+66.783830426 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls") pod "image-registry-6b86549d88-bbwrm" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0") : secret "image-registry-tls" not found Sep 18 21:06:48.235785 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:48.235750 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:06:48.235998 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:48.235901 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 21:06:48.235998 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:06:48.235981 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls podName:2321d814-a682-4e63-bc8b-c174d6cea7bc nodeName:}" failed. No retries permitted until 2026-09-18 21:07:04.235962983 +0000 UTC m=+66.884560568 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls") pod "dns-default-tsxlb" (UID: "2321d814-a682-4e63-bc8b-c174d6cea7bc") : secret "dns-default-metrics-tls" not found Sep 18 21:06:58.165512 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:06:58.165483 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fqsfb" Sep 18 21:07:03.547725 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.547677 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:07:03.548142 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.547747 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:07:03.554011 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.553986 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 21:07:03.554454 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.554437 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 21:07:03.557969 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:03.557950 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 18 21:07:03.558038 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:03.558027 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs podName:3ff122f5-c7e3-4cc6-b2a7-46b532f84d50 nodeName:}" failed. No retries permitted until 2026-09-18 21:08:07.558010079 +0000 UTC m=+130.206607648 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs") pod "network-metrics-daemon-bvvml" (UID: "3ff122f5-c7e3-4cc6-b2a7-46b532f84d50") : secret "metrics-daemon-secret" not found Sep 18 21:07:03.561251 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.561222 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbba6619-7498-4ef7-a9a8-174ffc7c3953-original-pull-secret\") pod \"global-pull-secret-syncer-hrxgz\" (UID: \"cbba6619-7498-4ef7-a9a8-174ffc7c3953\") " pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:07:03.749383 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.749344 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:07:03.753011 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.752989 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 21:07:03.763184 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.763159 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrxgz" Sep 18 21:07:03.763969 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.763951 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 21:07:03.773383 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.773362 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnvkl\" (UniqueName: \"kubernetes.io/projected/a565cb5c-0de6-4cf9-92de-a12b10b584ad-kube-api-access-dnvkl\") pod \"network-check-target-6cmhj\" (UID: \"a565cb5c-0de6-4cf9-92de-a12b10b584ad\") " pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:07:03.776552 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.776531 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7qn9\"" Sep 18 21:07:03.784456 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:03.784437 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:07:04.041724 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:04.041699 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hrxgz"] Sep 18 21:07:04.044371 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:07:04.044329 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbba6619_7498_4ef7_a9a8_174ffc7c3953.slice/crio-2219e43476f5bb219e043bc9e76250866a823b215195e2e600ac32f9d8b50688 WatchSource:0}: Error finding container 2219e43476f5bb219e043bc9e76250866a823b215195e2e600ac32f9d8b50688: Status 404 returned error can't find the container with id 2219e43476f5bb219e043bc9e76250866a823b215195e2e600ac32f9d8b50688 Sep 18 21:07:04.058491 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:04.058458 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6cmhj"] Sep 18 21:07:04.061046 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:07:04.060991 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda565cb5c_0de6_4cf9_92de_a12b10b584ad.slice/crio-5a48f57d69a82cb2e6caba8046290664f04ec204d00e3df0ab6200955e26abba WatchSource:0}: Error finding container 5a48f57d69a82cb2e6caba8046290664f04ec204d00e3df0ab6200955e26abba: Status 404 returned error can't find the container with id 5a48f57d69a82cb2e6caba8046290664f04ec204d00e3df0ab6200955e26abba Sep 18 21:07:04.152514 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:04.152473 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:07:04.152688 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:04.152625 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 21:07:04.152688 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:04.152646 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b86549d88-bbwrm: secret "image-registry-tls" not found Sep 18 21:07:04.152769 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:04.152701 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls podName:4251e8c2-55f8-488c-96f1-89cb5ffc93f0 nodeName:}" failed. No retries permitted until 2026-09-18 21:07:36.152684427 +0000 UTC m=+98.801281997 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls") pod "image-registry-6b86549d88-bbwrm" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0") : secret "image-registry-tls" not found Sep 18 21:07:04.252818 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:04.252788 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:07:04.252951 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:04.252896 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 21:07:04.252951 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:04.252944 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls podName:2321d814-a682-4e63-bc8b-c174d6cea7bc nodeName:}" failed. No retries permitted until 2026-09-18 21:07:36.252931414 +0000 UTC m=+98.901528983 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls") pod "dns-default-tsxlb" (UID: "2321d814-a682-4e63-bc8b-c174d6cea7bc") : secret "dns-default-metrics-tls" not found Sep 18 21:07:04.254884 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:04.254858 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6cmhj" event={"ID":"a565cb5c-0de6-4cf9-92de-a12b10b584ad","Type":"ContainerStarted","Data":"5a48f57d69a82cb2e6caba8046290664f04ec204d00e3df0ab6200955e26abba"} Sep 18 21:07:04.255929 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:04.255908 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hrxgz" event={"ID":"cbba6619-7498-4ef7-a9a8-174ffc7c3953","Type":"ContainerStarted","Data":"2219e43476f5bb219e043bc9e76250866a823b215195e2e600ac32f9d8b50688"} Sep 18 21:07:09.270917 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:09.270879 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hrxgz" event={"ID":"cbba6619-7498-4ef7-a9a8-174ffc7c3953","Type":"ContainerStarted","Data":"4e73817e176535840248b98b253165f6ebf72e679994a0b09c5b807f4346d560"} Sep 18 21:07:09.272411 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:09.272386 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6cmhj" event={"ID":"a565cb5c-0de6-4cf9-92de-a12b10b584ad","Type":"ContainerStarted","Data":"7f070f24983c8843091e5c9116d940051e935c1ce13ee9fe4158e759cd6958d0"} Sep 18 21:07:09.272514 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:09.272506 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:07:09.292929 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:09.292883 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hrxgz" podStartSLOduration=66.909520981 podStartE2EDuration="1m11.292866705s" totalImagesPullingTime="4.383345724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:07:09.291359966 +0000 UTC m=+71.939957558" watchObservedRunningTime="2026-09-18 21:07:09.292866705 +0000 UTC m=+71.941464320" Sep 18 21:07:09.320456 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:09.320402 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6cmhj" podStartSLOduration=66.959757055 podStartE2EDuration="1m11.320387319s" totalImagesPullingTime="4.360630264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:07:09.31997659 +0000 UTC m=+71.968574180" watchObservedRunningTime="2026-09-18 21:07:09.320387319 +0000 UTC m=+71.968984909" Sep 18 21:07:36.176152 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:36.176108 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") pod \"image-registry-6b86549d88-bbwrm\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:07:36.176756 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:36.176256 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 21:07:36.176756 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:36.176276 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b86549d88-bbwrm: secret "image-registry-tls" not found Sep 18 21:07:36.176756 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:36.176357 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls podName:4251e8c2-55f8-488c-96f1-89cb5ffc93f0 nodeName:}" failed. No retries permitted until 2026-09-18 21:08:40.176340212 +0000 UTC m=+162.824937781 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls") pod "image-registry-6b86549d88-bbwrm" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0") : secret "image-registry-tls" not found Sep 18 21:07:36.277369 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:36.277337 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:07:36.277528 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:36.277493 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 21:07:36.277568 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:36.277562 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls podName:2321d814-a682-4e63-bc8b-c174d6cea7bc nodeName:}" failed. No retries permitted until 2026-09-18 21:08:40.277545642 +0000 UTC m=+162.926143227 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls") pod "dns-default-tsxlb" (UID: "2321d814-a682-4e63-bc8b-c174d6cea7bc") : secret "dns-default-metrics-tls" not found Sep 18 21:07:37.681804 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.681769 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq"] Sep 18 21:07:37.685935 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.685918 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.689067 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.689040 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 18 21:07:37.689519 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.689501 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 18 21:07:37.689622 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.689593 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 18 21:07:37.689691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.689604 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 18 21:07:37.690501 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.690487 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-dfbwt\"" Sep 18 21:07:37.707044 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.707021 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq"] Sep 18 21:07:37.786899 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.786867 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2np4n\" (UniqueName: \"kubernetes.io/projected/e4021bfd-62e5-4e7c-8669-fad4ecdd0274-kube-api-access-2np4n\") pod \"service-ca-operator-57cdc4b768-2fkqq\" (UID: \"e4021bfd-62e5-4e7c-8669-fad4ecdd0274\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.786899 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.786911 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4021bfd-62e5-4e7c-8669-fad4ecdd0274-serving-cert\") pod \"service-ca-operator-57cdc4b768-2fkqq\" (UID: \"e4021bfd-62e5-4e7c-8669-fad4ecdd0274\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.787139 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.786940 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e4021bfd-62e5-4e7c-8669-fad4ecdd0274-config\") pod \"service-ca-operator-57cdc4b768-2fkqq\" (UID: \"e4021bfd-62e5-4e7c-8669-fad4ecdd0274\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.887712 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.887676 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2np4n\" (UniqueName: \"kubernetes.io/projected/e4021bfd-62e5-4e7c-8669-fad4ecdd0274-kube-api-access-2np4n\") pod \"service-ca-operator-57cdc4b768-2fkqq\" (UID: \"e4021bfd-62e5-4e7c-8669-fad4ecdd0274\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.887712 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.887715 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4021bfd-62e5-4e7c-8669-fad4ecdd0274-serving-cert\") pod \"service-ca-operator-57cdc4b768-2fkqq\" (UID: \"e4021bfd-62e5-4e7c-8669-fad4ecdd0274\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.887963 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.887743 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e4021bfd-62e5-4e7c-8669-fad4ecdd0274-config\") pod \"service-ca-operator-57cdc4b768-2fkqq\" (UID: \"e4021bfd-62e5-4e7c-8669-fad4ecdd0274\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.888236 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.888217 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e4021bfd-62e5-4e7c-8669-fad4ecdd0274-config\") pod \"service-ca-operator-57cdc4b768-2fkqq\" (UID: \"e4021bfd-62e5-4e7c-8669-fad4ecdd0274\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.889957 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.889925 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e4021bfd-62e5-4e7c-8669-fad4ecdd0274-serving-cert\") pod \"service-ca-operator-57cdc4b768-2fkqq\" (UID: \"e4021bfd-62e5-4e7c-8669-fad4ecdd0274\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.920464 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.920427 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2np4n\" (UniqueName: \"kubernetes.io/projected/e4021bfd-62e5-4e7c-8669-fad4ecdd0274-kube-api-access-2np4n\") pod \"service-ca-operator-57cdc4b768-2fkqq\" (UID: \"e4021bfd-62e5-4e7c-8669-fad4ecdd0274\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:37.994413 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:37.994322 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" Sep 18 21:07:38.277868 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.277784 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq"] Sep 18 21:07:38.280893 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:07:38.280863 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4021bfd_62e5_4e7c_8669_fad4ecdd0274.slice/crio-6427f9e217c089bfc3f830215025c61ac20b76a800ff047324a6feaf1b908ac9 WatchSource:0}: Error finding container 6427f9e217c089bfc3f830215025c61ac20b76a800ff047324a6feaf1b908ac9: Status 404 returned error can't find the container with id 6427f9e217c089bfc3f830215025c61ac20b76a800ff047324a6feaf1b908ac9 Sep 18 21:07:38.346032 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.345996 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" event={"ID":"e4021bfd-62e5-4e7c-8669-fad4ecdd0274","Type":"ContainerStarted","Data":"6427f9e217c089bfc3f830215025c61ac20b76a800ff047324a6feaf1b908ac9"} Sep 18 21:07:38.759420 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.759382 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-hbflc"] Sep 18 21:07:38.762154 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.762129 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:38.765188 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.765162 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vg4st\"" Sep 18 21:07:38.765340 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.765215 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 18 21:07:38.766027 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.766004 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 18 21:07:38.774202 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.774176 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-hbflc"] Sep 18 21:07:38.896462 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.896425 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:38.896643 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.896571 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c99ba6d4-0611-419c-87e4-e927682a92b8-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:38.997965 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.997924 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:38.998165 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.998065 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c99ba6d4-0611-419c-87e4-e927682a92b8-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:38.998165 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:38.998091 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 18 21:07:38.998279 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:38.998166 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert podName:c99ba6d4-0611-419c-87e4-e927682a92b8 nodeName:}" failed. No retries permitted until 2026-09-18 21:07:39.498148774 +0000 UTC m=+102.146746355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-hbflc" (UID: "c99ba6d4-0611-419c-87e4-e927682a92b8") : secret "networking-console-plugin-cert" not found Sep 18 21:07:38.998754 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:38.998729 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c99ba6d4-0611-419c-87e4-e927682a92b8-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:39.502595 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:39.502555 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:39.502792 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:39.502716 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 18 21:07:39.502851 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:39.502799 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert podName:c99ba6d4-0611-419c-87e4-e927682a92b8 nodeName:}" failed. No retries permitted until 2026-09-18 21:07:40.502776684 +0000 UTC m=+103.151374273 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-hbflc" (UID: "c99ba6d4-0611-419c-87e4-e927682a92b8") : secret "networking-console-plugin-cert" not found Sep 18 21:07:40.277940 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:40.277912 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6cmhj" Sep 18 21:07:40.353702 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:40.353666 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" event={"ID":"e4021bfd-62e5-4e7c-8669-fad4ecdd0274","Type":"ContainerStarted","Data":"a8fc760d004beda2197669e7f7622088a99fba1a744538f2c47fb0692e50d264"} Sep 18 21:07:40.384546 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:40.384487 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" podStartSLOduration=1.511091781 podStartE2EDuration="3.384464357s" totalImagesPullingTime="1.873372576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:07:40.383194398 +0000 UTC m=+103.031791988" watchObservedRunningTime="2026-09-18 21:07:40.384464357 +0000 UTC m=+103.033061949" Sep 18 21:07:40.511630 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:40.511540 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:40.511779 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:40.511659 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 18 21:07:40.511779 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:40.511713 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert podName:c99ba6d4-0611-419c-87e4-e927682a92b8 nodeName:}" failed. No retries permitted until 2026-09-18 21:07:42.511699826 +0000 UTC m=+105.160297395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-hbflc" (UID: "c99ba6d4-0611-419c-87e4-e927682a92b8") : secret "networking-console-plugin-cert" not found Sep 18 21:07:41.581963 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.581929 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mhgzm"] Sep 18 21:07:41.603413 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.603382 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.606953 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.606858 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 18 21:07:41.607120 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.607038 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 18 21:07:41.607510 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.607301 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 18 21:07:41.607510 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.607363 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 18 21:07:41.607510 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.607424 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6nnv\"" Sep 18 21:07:41.607700 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.607620 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mhgzm"] Sep 18 21:07:41.721590 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.721545 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22d79160-d48c-4229-9681-e10eefd24c2b-crio-socket\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.721775 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.721622 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.721775 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.721643 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjp5r\" (UniqueName: \"kubernetes.io/projected/22d79160-d48c-4229-9681-e10eefd24c2b-kube-api-access-tjp5r\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.721775 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.721670 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.721775 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.721692 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22d79160-d48c-4229-9681-e10eefd24c2b-data-volume\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.822991 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.822931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjp5r\" (UniqueName: \"kubernetes.io/projected/22d79160-d48c-4229-9681-e10eefd24c2b-kube-api-access-tjp5r\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.823171 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.823014 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.823171 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.823043 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22d79160-d48c-4229-9681-e10eefd24c2b-data-volume\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.823171 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:41.823137 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 21:07:41.823336 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.823215 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22d79160-d48c-4229-9681-e10eefd24c2b-crio-socket\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.823336 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:41.823224 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:42.323202853 +0000 UTC m=+104.971800442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : configmap "kube-rbac-proxy" not found Sep 18 21:07:41.823336 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.823143 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22d79160-d48c-4229-9681-e10eefd24c2b-crio-socket\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.823336 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.823283 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.823485 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:41.823387 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 21:07:41.823485 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:41.823440 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:42.323428114 +0000 UTC m=+104.972025683 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : secret "insights-runtime-extractor-tls" not found Sep 18 21:07:41.823553 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.823496 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22d79160-d48c-4229-9681-e10eefd24c2b-data-volume\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:41.832220 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:41.832177 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjp5r\" (UniqueName: \"kubernetes.io/projected/22d79160-d48c-4229-9681-e10eefd24c2b-kube-api-access-tjp5r\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:42.328126 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:42.328084 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:42.328307 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:42.328138 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:42.328307 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:42.328233 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 21:07:42.328307 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:42.328241 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 21:07:42.328307 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:42.328286 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:43.328271733 +0000 UTC m=+105.976869302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : configmap "kube-rbac-proxy" not found Sep 18 21:07:42.328307 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:42.328298 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:43.328292627 +0000 UTC m=+105.976890195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : secret "insights-runtime-extractor-tls" not found Sep 18 21:07:42.530207 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:42.530168 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:42.530411 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:42.530297 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 18 21:07:42.530411 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:42.530369 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert podName:c99ba6d4-0611-419c-87e4-e927682a92b8 nodeName:}" failed. No retries permitted until 2026-09-18 21:07:46.530354339 +0000 UTC m=+109.178951909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-hbflc" (UID: "c99ba6d4-0611-419c-87e4-e927682a92b8") : secret "networking-console-plugin-cert" not found Sep 18 21:07:43.304333 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.304285 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pj68m_626b4d06-d3b0-44fb-94b9-94f3131d6afc/dns-node-resolver/0.log" Sep 18 21:07:43.337611 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.337571 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:43.337783 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.337636 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:43.337783 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:43.337732 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 21:07:43.337894 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:43.337785 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 21:07:43.337894 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:43.337796 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:45.337779124 +0000 UTC m=+107.986376707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : secret "insights-runtime-extractor-tls" not found Sep 18 21:07:43.337894 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:43.337865 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:45.337844212 +0000 UTC m=+107.986441787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : configmap "kube-rbac-proxy" not found Sep 18 21:07:43.513529 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.513498 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-47st7"] Sep 18 21:07:43.528859 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.528834 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.532008 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.531983 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 18 21:07:43.532139 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.532058 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 18 21:07:43.533028 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.533010 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 18 21:07:43.533149 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.533019 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 18 21:07:43.534541 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.534519 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jtfbd\"" Sep 18 21:07:43.535341 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.535299 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-47st7"] Sep 18 21:07:43.640544 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.640507 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e710260a-311c-4276-a886-a7e9bfa0cba6-signing-key\") pod \"service-ca-55f6fc7b86-47st7\" (UID: \"e710260a-311c-4276-a886-a7e9bfa0cba6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.640774 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.640571 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e710260a-311c-4276-a886-a7e9bfa0cba6-signing-cabundle\") pod \"service-ca-55f6fc7b86-47st7\" (UID: \"e710260a-311c-4276-a886-a7e9bfa0cba6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.640774 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.640660 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8cw9\" (UniqueName: \"kubernetes.io/projected/e710260a-311c-4276-a886-a7e9bfa0cba6-kube-api-access-t8cw9\") pod \"service-ca-55f6fc7b86-47st7\" (UID: \"e710260a-311c-4276-a886-a7e9bfa0cba6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.741580 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.741542 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8cw9\" (UniqueName: \"kubernetes.io/projected/e710260a-311c-4276-a886-a7e9bfa0cba6-kube-api-access-t8cw9\") pod \"service-ca-55f6fc7b86-47st7\" (UID: \"e710260a-311c-4276-a886-a7e9bfa0cba6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.741746 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.741636 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e710260a-311c-4276-a886-a7e9bfa0cba6-signing-key\") pod \"service-ca-55f6fc7b86-47st7\" (UID: \"e710260a-311c-4276-a886-a7e9bfa0cba6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.741746 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.741689 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e710260a-311c-4276-a886-a7e9bfa0cba6-signing-cabundle\") pod \"service-ca-55f6fc7b86-47st7\" (UID: \"e710260a-311c-4276-a886-a7e9bfa0cba6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.742419 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.742401 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e710260a-311c-4276-a886-a7e9bfa0cba6-signing-cabundle\") pod \"service-ca-55f6fc7b86-47st7\" (UID: \"e710260a-311c-4276-a886-a7e9bfa0cba6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.744173 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.744151 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e710260a-311c-4276-a886-a7e9bfa0cba6-signing-key\") pod \"service-ca-55f6fc7b86-47st7\" (UID: \"e710260a-311c-4276-a886-a7e9bfa0cba6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.750059 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.750038 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8cw9\" (UniqueName: \"kubernetes.io/projected/e710260a-311c-4276-a886-a7e9bfa0cba6-kube-api-access-t8cw9\") pod \"service-ca-55f6fc7b86-47st7\" (UID: \"e710260a-311c-4276-a886-a7e9bfa0cba6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:43.838646 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:43.838616 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" Sep 18 21:07:44.143711 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:44.143665 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-47st7"] Sep 18 21:07:44.149251 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:07:44.149221 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode710260a_311c_4276_a886_a7e9bfa0cba6.slice/crio-85c1a0a1020435ecebfdc058e07f96f3cf9e3e9c8d0f9bd44d436ac00ab0c9ef WatchSource:0}: Error finding container 85c1a0a1020435ecebfdc058e07f96f3cf9e3e9c8d0f9bd44d436ac00ab0c9ef: Status 404 returned error can't find the container with id 85c1a0a1020435ecebfdc058e07f96f3cf9e3e9c8d0f9bd44d436ac00ab0c9ef Sep 18 21:07:44.364947 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:44.364912 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" event={"ID":"e710260a-311c-4276-a886-a7e9bfa0cba6","Type":"ContainerStarted","Data":"d03688934d2f99ef898f4b70f8a72e141dd261294bcdba00dbd1540e3686e403"} Sep 18 21:07:44.365359 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:44.364972 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" event={"ID":"e710260a-311c-4276-a886-a7e9bfa0cba6","Type":"ContainerStarted","Data":"85c1a0a1020435ecebfdc058e07f96f3cf9e3e9c8d0f9bd44d436ac00ab0c9ef"} Sep 18 21:07:44.388829 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:44.388772 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-47st7" podStartSLOduration=1.388755019 podStartE2EDuration="1.388755019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:07:44.387541807 +0000 UTC m=+107.036139399" watchObservedRunningTime="2026-09-18 21:07:44.388755019 +0000 UTC m=+107.037352610" Sep 18 21:07:44.710715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:44.710690 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pvrc4_28264d96-1e7c-49bf-9654-5ad9c675d9b6/node-ca/0.log" Sep 18 21:07:45.103575 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:45.103549 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5n7pb_37a4721d-1c7f-434e-97d0-f1d32baae843/serve-healthcheck-canary/0.log" Sep 18 21:07:45.355297 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:45.355197 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:45.355297 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:45.355247 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:45.355576 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:45.355353 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 21:07:45.355576 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:45.355407 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:49.355394092 +0000 UTC m=+112.003991660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : configmap "kube-rbac-proxy" not found Sep 18 21:07:45.355576 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:45.355407 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 21:07:45.355576 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:45.355477 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:49.355458353 +0000 UTC m=+112.004055936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : secret "insights-runtime-extractor-tls" not found Sep 18 21:07:46.564919 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:46.564875 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:46.565362 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:46.565025 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 18 21:07:46.565362 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:46.565093 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert podName:c99ba6d4-0611-419c-87e4-e927682a92b8 nodeName:}" failed. No retries permitted until 2026-09-18 21:07:54.565077238 +0000 UTC m=+117.213674807 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-hbflc" (UID: "c99ba6d4-0611-419c-87e4-e927682a92b8") : secret "networking-console-plugin-cert" not found Sep 18 21:07:49.386474 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:49.386436 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:49.386860 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:49.386558 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:49.386860 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:49.386561 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 21:07:49.386860 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:49.386635 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 21:07:49.386860 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:49.386652 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:57.386632855 +0000 UTC m=+120.035230444 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : configmap "kube-rbac-proxy" not found Sep 18 21:07:49.386860 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:07:49.386682 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls podName:22d79160-d48c-4229-9681-e10eefd24c2b nodeName:}" failed. No retries permitted until 2026-09-18 21:07:57.386673074 +0000 UTC m=+120.035270643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhgzm" (UID: "22d79160-d48c-4229-9681-e10eefd24c2b") : secret "insights-runtime-extractor-tls" not found Sep 18 21:07:54.627748 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:54.627704 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:54.630003 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:54.629982 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c99ba6d4-0611-419c-87e4-e927682a92b8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-hbflc\" (UID: \"c99ba6d4-0611-419c-87e4-e927682a92b8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:54.673136 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:54.673093 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" Sep 18 21:07:54.944136 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:54.944056 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-hbflc"] Sep 18 21:07:54.946910 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:07:54.946886 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99ba6d4_0611_419c_87e4_e927682a92b8.slice/crio-5220a097ebc3b46a46e6b8da0f731bb022d5e6e06c325e18e78bef92df8641c7 WatchSource:0}: Error finding container 5220a097ebc3b46a46e6b8da0f731bb022d5e6e06c325e18e78bef92df8641c7: Status 404 returned error can't find the container with id 5220a097ebc3b46a46e6b8da0f731bb022d5e6e06c325e18e78bef92df8641c7 Sep 18 21:07:55.391637 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:55.391600 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" event={"ID":"c99ba6d4-0611-419c-87e4-e927682a92b8","Type":"ContainerStarted","Data":"5220a097ebc3b46a46e6b8da0f731bb022d5e6e06c325e18e78bef92df8641c7"} Sep 18 21:07:56.396504 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:56.396461 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" event={"ID":"c99ba6d4-0611-419c-87e4-e927682a92b8","Type":"ContainerStarted","Data":"8bf174d9673ab4436fa55a3536f657e0e019bbb1087f65fce132faf56bcb87e0"} Sep 18 21:07:56.419179 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:56.419131 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-hbflc" podStartSLOduration=17.385581235 podStartE2EDuration="18.419114048s" totalImagesPullingTime="1.033532813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:07:56.418001802 +0000 UTC m=+119.066599392" watchObservedRunningTime="2026-09-18 21:07:56.419114048 +0000 UTC m=+119.067711642" Sep 18 21:07:57.452178 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:57.452138 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:57.452178 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:57.452181 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:57.452686 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:57.452667 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22d79160-d48c-4229-9681-e10eefd24c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:57.454518 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:57.454497 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22d79160-d48c-4229-9681-e10eefd24c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhgzm\" (UID: \"22d79160-d48c-4229-9681-e10eefd24c2b\") " pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:57.512886 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:57.512851 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mhgzm" Sep 18 21:07:57.793976 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:57.793873 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mhgzm"] Sep 18 21:07:57.797249 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:07:57.797221 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d79160_d48c_4229_9681_e10eefd24c2b.slice/crio-b089e1a4f4063d72d2776dd352d0d51639f4bba6ba3102820d5d1ca203c2b7cb WatchSource:0}: Error finding container b089e1a4f4063d72d2776dd352d0d51639f4bba6ba3102820d5d1ca203c2b7cb: Status 404 returned error can't find the container with id b089e1a4f4063d72d2776dd352d0d51639f4bba6ba3102820d5d1ca203c2b7cb Sep 18 21:07:58.404079 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:58.404052 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mhgzm" event={"ID":"22d79160-d48c-4229-9681-e10eefd24c2b","Type":"ContainerStarted","Data":"29116f465c3157db8c31cf5e41e50cdb262086681676172dc0f91294f4264589"} Sep 18 21:07:58.404168 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:58.404111 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mhgzm" event={"ID":"22d79160-d48c-4229-9681-e10eefd24c2b","Type":"ContainerStarted","Data":"b089e1a4f4063d72d2776dd352d0d51639f4bba6ba3102820d5d1ca203c2b7cb"} Sep 18 21:07:59.409592 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:07:59.409555 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mhgzm" event={"ID":"22d79160-d48c-4229-9681-e10eefd24c2b","Type":"ContainerStarted","Data":"8997a13a9495a797244a9e2971068d4e0f100d1d1f9d969481374331c9cb4677"} Sep 18 21:08:00.415998 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:00.415963 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mhgzm" event={"ID":"22d79160-d48c-4229-9681-e10eefd24c2b","Type":"ContainerStarted","Data":"00d92649a7e873f77f13ac4d77d64b928c429554e2a3f9989450bb1f1c9edfb6"} Sep 18 21:08:00.448066 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:00.448011 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mhgzm" podStartSLOduration=17.671041918 podStartE2EDuration="19.447993966s" totalImagesPullingTime="1.776952048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:07:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:08:00.446239597 +0000 UTC m=+123.094837187" watchObservedRunningTime="2026-09-18 21:08:00.447993966 +0000 UTC m=+123.096591553" Sep 18 21:08:06.014626 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.014587 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq"] Sep 18 21:08:06.016792 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.016776 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.021861 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.021828 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 18 21:08:06.022828 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.022631 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 18 21:08:06.022828 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.022638 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 18 21:08:06.022828 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.022732 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 18 21:08:06.022828 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.022700 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 18 21:08:06.023350 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.023021 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 18 21:08:06.023350 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.023024 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 18 21:08:06.122562 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.122528 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.122718 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.122588 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-ca\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.122718 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.122615 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.122718 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.122635 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-hub\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.122840 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.122753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2wp4\" (UniqueName: \"kubernetes.io/projected/3ab5d99f-4e7c-417c-a285-531f1b6248f3-kube-api-access-x2wp4\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.122840 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.122790 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ab5d99f-4e7c-417c-a285-531f1b6248f3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.131184 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.131150 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq"] Sep 18 21:08:06.223937 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.223902 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-hub\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.224146 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.223969 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2wp4\" (UniqueName: \"kubernetes.io/projected/3ab5d99f-4e7c-417c-a285-531f1b6248f3-kube-api-access-x2wp4\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.224146 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.224000 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ab5d99f-4e7c-417c-a285-531f1b6248f3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.224146 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.224024 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.224146 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.224094 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-ca\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.224146 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.224131 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.224829 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.224805 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ab5d99f-4e7c-417c-a285-531f1b6248f3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.226631 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.226603 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-ca\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.226731 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.226667 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.226790 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.226774 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.226853 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.226838 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3ab5d99f-4e7c-417c-a285-531f1b6248f3-hub\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.261219 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.261174 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2wp4\" (UniqueName: \"kubernetes.io/projected/3ab5d99f-4e7c-417c-a285-531f1b6248f3-kube-api-access-x2wp4\") pod \"cluster-proxy-proxy-agent-6455fc8956-4ngwq\" (UID: \"3ab5d99f-4e7c-417c-a285-531f1b6248f3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.333881 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.333785 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" Sep 18 21:08:06.656424 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:06.656393 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq"] Sep 18 21:08:06.659393 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:08:06.659368 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ab5d99f_4e7c_417c_a285_531f1b6248f3.slice/crio-991c7e67089bbcb45a64cdbafc6c5afe2b1eb48de0e9f2ee8fd216db9107fccb WatchSource:0}: Error finding container 991c7e67089bbcb45a64cdbafc6c5afe2b1eb48de0e9f2ee8fd216db9107fccb: Status 404 returned error can't find the container with id 991c7e67089bbcb45a64cdbafc6c5afe2b1eb48de0e9f2ee8fd216db9107fccb Sep 18 21:08:07.438309 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:07.438267 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" event={"ID":"3ab5d99f-4e7c-417c-a285-531f1b6248f3","Type":"ContainerStarted","Data":"991c7e67089bbcb45a64cdbafc6c5afe2b1eb48de0e9f2ee8fd216db9107fccb"} Sep 18 21:08:07.634756 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:07.634720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:08:07.637887 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:07.637844 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff122f5-c7e3-4cc6-b2a7-46b532f84d50-metrics-certs\") pod \"network-metrics-daemon-bvvml\" (UID: \"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50\") " pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:08:07.672407 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:07.672376 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pzg28\"" Sep 18 21:08:07.679865 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:07.679689 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvvml" Sep 18 21:08:07.977486 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:07.977398 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bvvml"] Sep 18 21:08:07.981866 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:08:07.981831 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff122f5_c7e3_4cc6_b2a7_46b532f84d50.slice/crio-a51d60e3c2f9677c2c02fddb9b99d7b1efbe5b4cab9ba05e6b163f8cfd3e147c WatchSource:0}: Error finding container a51d60e3c2f9677c2c02fddb9b99d7b1efbe5b4cab9ba05e6b163f8cfd3e147c: Status 404 returned error can't find the container with id a51d60e3c2f9677c2c02fddb9b99d7b1efbe5b4cab9ba05e6b163f8cfd3e147c Sep 18 21:08:08.443366 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:08.443306 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bvvml" event={"ID":"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50","Type":"ContainerStarted","Data":"a51d60e3c2f9677c2c02fddb9b99d7b1efbe5b4cab9ba05e6b163f8cfd3e147c"} Sep 18 21:08:11.455191 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:11.455152 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" event={"ID":"3ab5d99f-4e7c-417c-a285-531f1b6248f3","Type":"ContainerStarted","Data":"16292d854eb5a6664b0998a385923bab49ecbfc984bb826b2463654366e5abd8"} Sep 18 21:08:11.455692 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:11.455201 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" event={"ID":"3ab5d99f-4e7c-417c-a285-531f1b6248f3","Type":"ContainerStarted","Data":"3502a0099baf8ccabad11742efcde21446882c167e1212bdaed089f03469960a"} Sep 18 21:08:11.455692 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:11.455221 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" event={"ID":"3ab5d99f-4e7c-417c-a285-531f1b6248f3","Type":"ContainerStarted","Data":"7ac2d3c9f1cbccda5e4622f13850334d0ff9ae3d1318769a192a3fb75447f03e"} Sep 18 21:08:11.457012 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:11.456988 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bvvml" event={"ID":"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50","Type":"ContainerStarted","Data":"50dbd6e8b4aa06f305d34c4aeba0d2d70d330c01d93ab5b140cf29375304c3ba"} Sep 18 21:08:11.457088 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:11.457033 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bvvml" event={"ID":"3ff122f5-c7e3-4cc6-b2a7-46b532f84d50","Type":"ContainerStarted","Data":"ba0128b3e2002a484763fddc599dae0442ea6e54f0600b994bb815a8cea9008e"} Sep 18 21:08:11.483903 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:11.483847 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6455fc8956-4ngwq" podStartSLOduration=2.709713419 podStartE2EDuration="6.4838317s" totalImagesPullingTime="3.774118281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:08:11.482126154 +0000 UTC m=+134.130723745" watchObservedRunningTime="2026-09-18 21:08:11.4838317 +0000 UTC m=+134.132429291" Sep 18 21:08:11.509159 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:11.509109 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bvvml" podStartSLOduration=131.070671242 podStartE2EDuration="2m13.509095167s" totalImagesPullingTime="2.438423925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:08:11.508387593 +0000 UTC m=+134.156985185" watchObservedRunningTime="2026-09-18 21:08:11.509095167 +0000 UTC m=+134.157692757" Sep 18 21:08:15.127465 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.127433 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dps6z"] Sep 18 21:08:15.130716 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.130695 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.134752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.134731 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 18 21:08:15.135144 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.135122 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 18 21:08:15.135519 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.135496 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 18 21:08:15.135629 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.135520 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 18 21:08:15.135629 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.135542 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 18 21:08:15.135866 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.135851 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 18 21:08:15.135950 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.135930 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6tklw\"" Sep 18 21:08:15.187668 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.187632 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-textfile\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.187865 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.187687 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.187865 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.187715 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-accelerators-collector-config\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.187865 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.187765 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3382c007-d5c3-4dc8-8e6c-994884dbb284-sys\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.187865 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.187787 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-wtmp\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.187865 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.187841 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmq7s\" (UniqueName: \"kubernetes.io/projected/3382c007-d5c3-4dc8-8e6c-994884dbb284-kube-api-access-wmq7s\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.188122 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.187926 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-tls\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.188122 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.187957 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3382c007-d5c3-4dc8-8e6c-994884dbb284-metrics-client-ca\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.188122 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.187989 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3382c007-d5c3-4dc8-8e6c-994884dbb284-root\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.288754 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.288720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmq7s\" (UniqueName: \"kubernetes.io/projected/3382c007-d5c3-4dc8-8e6c-994884dbb284-kube-api-access-wmq7s\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.288960 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.288781 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-tls\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.288960 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.288889 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3382c007-d5c3-4dc8-8e6c-994884dbb284-metrics-client-ca\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.288960 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.288912 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3382c007-d5c3-4dc8-8e6c-994884dbb284-root\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.288960 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.288934 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-textfile\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289165 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.288970 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289165 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.288996 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-accelerators-collector-config\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289165 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.289027 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3382c007-d5c3-4dc8-8e6c-994884dbb284-sys\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289165 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.289053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-wtmp\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289385 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.289191 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-wtmp\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289385 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.289026 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3382c007-d5c3-4dc8-8e6c-994884dbb284-root\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289385 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.289207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3382c007-d5c3-4dc8-8e6c-994884dbb284-sys\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289760 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.289673 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-accelerators-collector-config\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289760 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.289726 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-textfile\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.289897 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.289765 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3382c007-d5c3-4dc8-8e6c-994884dbb284-metrics-client-ca\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.291389 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.291357 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.291670 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.291651 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3382c007-d5c3-4dc8-8e6c-994884dbb284-node-exporter-tls\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.302044 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.302018 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmq7s\" (UniqueName: \"kubernetes.io/projected/3382c007-d5c3-4dc8-8e6c-994884dbb284-kube-api-access-wmq7s\") pod \"node-exporter-dps6z\" (UID: \"3382c007-d5c3-4dc8-8e6c-994884dbb284\") " pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.441013 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.440924 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dps6z" Sep 18 21:08:15.452975 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:08:15.452937 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3382c007_d5c3_4dc8_8e6c_994884dbb284.slice/crio-545b8fd6db62e5b339745bf780326f26fc9f334687526660a3735cd5ba831c92 WatchSource:0}: Error finding container 545b8fd6db62e5b339745bf780326f26fc9f334687526660a3735cd5ba831c92: Status 404 returned error can't find the container with id 545b8fd6db62e5b339745bf780326f26fc9f334687526660a3735cd5ba831c92 Sep 18 21:08:15.472357 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:15.472302 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dps6z" event={"ID":"3382c007-d5c3-4dc8-8e6c-994884dbb284","Type":"ContainerStarted","Data":"545b8fd6db62e5b339745bf780326f26fc9f334687526660a3735cd5ba831c92"} Sep 18 21:08:16.477351 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:16.477300 2557 generic.go:356] "Generic (PLEG): container finished" podID="3382c007-d5c3-4dc8-8e6c-994884dbb284" containerID="762242359e2bf7cb7978061187b3a5a090e0bbf4e02b0dcb5b01e2f7ea84bc17" exitCode=0 Sep 18 21:08:16.477730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:16.477354 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dps6z" event={"ID":"3382c007-d5c3-4dc8-8e6c-994884dbb284","Type":"ContainerDied","Data":"762242359e2bf7cb7978061187b3a5a090e0bbf4e02b0dcb5b01e2f7ea84bc17"} Sep 18 21:08:17.483137 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:17.483106 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dps6z" event={"ID":"3382c007-d5c3-4dc8-8e6c-994884dbb284","Type":"ContainerStarted","Data":"bceba77e8bb3e4417074dc54d98a96e5c5bbb6dcd10826d31337df58f1d410bd"} Sep 18 21:08:17.483529 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:17.483155 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dps6z" event={"ID":"3382c007-d5c3-4dc8-8e6c-994884dbb284","Type":"ContainerStarted","Data":"e8be9bd3df454bca67b5624a32061f7bfb0de4b812ff10a79defd280daab2bee"} Sep 18 21:08:17.535823 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:17.535770 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dps6z" podStartSLOduration=1.6591939789999999 podStartE2EDuration="2.53575312s" totalImagesPullingTime="816.401016ms" totalInitContainerRuntime="60.158125ms" isStatefulPod=true podCreationTimestamp="2026-09-18 21:08:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:08:17.5353758 +0000 UTC m=+140.183973391" watchObservedRunningTime="2026-09-18 21:08:17.53575312 +0000 UTC m=+140.184350711" Sep 18 21:08:19.596535 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.596499 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-89cb5fbc4-svcnv"] Sep 18 21:08:19.599647 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.599632 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.603629 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:19.603600 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-client-certs\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" type="*v1.Secret" Sep 18 21:08:19.603991 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:19.603973 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-dockercfg-qhrkj\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qhrkj\"" type="*v1.Secret" Sep 18 21:08:19.604289 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:19.604262 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"metrics-server-89cb5fbc4-svcnv\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="31cbff63-8b3a-420d-a9b3-9310f05cbcd3" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.604373 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:19.604354 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"metrics-server-audit-profiles\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" type="*v1.ConfigMap" Sep 18 21:08:19.604433 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:19.604371 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kubelet-serving-ca-bundle\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" type="*v1.ConfigMap" Sep 18 21:08:19.604433 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:19.604377 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-tls\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" type="*v1.Secret" Sep 18 21:08:19.604433 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:19.604385 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-29mcd0hu3r218\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-29mcd0hu3r218\"" type="*v1.Secret" Sep 18 21:08:19.623782 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.623755 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-client-certs\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.623893 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.623791 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-metrics-server-audit-profiles\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.623893 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.623823 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-client-ca-bundle\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.623962 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.623889 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.623962 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.623933 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjf8j\" (UniqueName: \"kubernetes.io/projected/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-kube-api-access-vjf8j\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.623962 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.623955 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-tls\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.624053 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.623989 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-audit-log\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.633239 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.633212 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-89cb5fbc4-svcnv"] Sep 18 21:08:19.724493 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.724458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjf8j\" (UniqueName: \"kubernetes.io/projected/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-kube-api-access-vjf8j\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.724493 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.724496 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-tls\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.724730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.724528 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-audit-log\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.724730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.724570 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-client-certs\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.724730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.724593 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-metrics-server-audit-profiles\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.724730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.724625 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-client-ca-bundle\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.724730 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.724670 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.725006 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.724985 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-audit-log\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:19.738375 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:19.738344 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjf8j\" (UniqueName: \"kubernetes.io/projected/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-kube-api-access-vjf8j\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:20.547216 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:20.547180 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qhrkj\"" Sep 18 21:08:20.724941 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.724897 2557 configmap.go:193] Couldn't get configMap openshift-monitoring/kubelet-serving-ca-bundle: failed to sync configmap cache: timed out waiting for the condition Sep 18 21:08:20.724941 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.724922 2557 secret.go:189] Couldn't get secret openshift-monitoring/metrics-server-client-certs: failed to sync secret cache: timed out waiting for the condition Sep 18 21:08:20.724941 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.724929 2557 secret.go:189] Couldn't get secret openshift-monitoring/metrics-server-29mcd0hu3r218: failed to sync secret cache: timed out waiting for the condition Sep 18 21:08:20.725412 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.724904 2557 configmap.go:193] Couldn't get configMap openshift-monitoring/metrics-server-audit-profiles: failed to sync configmap cache: timed out waiting for the condition Sep 18 21:08:20.725412 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.724914 2557 secret.go:189] Couldn't get secret openshift-monitoring/metrics-server-tls: failed to sync secret cache: timed out waiting for the condition Sep 18 21:08:20.725412 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.724994 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-configmap-kubelet-serving-ca-bundle podName:31cbff63-8b3a-420d-a9b3-9310f05cbcd3 nodeName:}" failed. No retries permitted until 2026-09-18 21:08:21.224972715 +0000 UTC m=+143.873570283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "configmap-kubelet-serving-ca-bundle" (UniqueName: "kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-configmap-kubelet-serving-ca-bundle") pod "metrics-server-89cb5fbc4-svcnv" (UID: "31cbff63-8b3a-420d-a9b3-9310f05cbcd3") : failed to sync configmap cache: timed out waiting for the condition Sep 18 21:08:20.725412 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.725012 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-client-certs podName:31cbff63-8b3a-420d-a9b3-9310f05cbcd3 nodeName:}" failed. No retries permitted until 2026-09-18 21:08:21.225004495 +0000 UTC m=+143.873602064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-metrics-server-client-certs" (UniqueName: "kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-client-certs") pod "metrics-server-89cb5fbc4-svcnv" (UID: "31cbff63-8b3a-420d-a9b3-9310f05cbcd3") : failed to sync secret cache: timed out waiting for the condition Sep 18 21:08:20.725412 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.725024 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-client-ca-bundle podName:31cbff63-8b3a-420d-a9b3-9310f05cbcd3 nodeName:}" failed. No retries permitted until 2026-09-18 21:08:21.225018659 +0000 UTC m=+143.873616227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "client-ca-bundle" (UniqueName: "kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-client-ca-bundle") pod "metrics-server-89cb5fbc4-svcnv" (UID: "31cbff63-8b3a-420d-a9b3-9310f05cbcd3") : failed to sync secret cache: timed out waiting for the condition Sep 18 21:08:20.725412 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.725033 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-metrics-server-audit-profiles podName:31cbff63-8b3a-420d-a9b3-9310f05cbcd3 nodeName:}" failed. No retries permitted until 2026-09-18 21:08:21.225027946 +0000 UTC m=+143.873625515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-server-audit-profiles" (UniqueName: "kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-metrics-server-audit-profiles") pod "metrics-server-89cb5fbc4-svcnv" (UID: "31cbff63-8b3a-420d-a9b3-9310f05cbcd3") : failed to sync configmap cache: timed out waiting for the condition Sep 18 21:08:20.725412 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:20.725042 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-tls podName:31cbff63-8b3a-420d-a9b3-9310f05cbcd3 nodeName:}" failed. No retries permitted until 2026-09-18 21:08:21.225036869 +0000 UTC m=+143.873634438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-metrics-server-tls" (UniqueName: "kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-tls") pod "metrics-server-89cb5fbc4-svcnv" (UID: "31cbff63-8b3a-420d-a9b3-9310f05cbcd3") : failed to sync secret cache: timed out waiting for the condition Sep 18 21:08:20.727034 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:20.727017 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 18 21:08:20.761613 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:20.761584 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 18 21:08:20.776749 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:20.776725 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-29mcd0hu3r218\"" Sep 18 21:08:20.805289 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:20.805224 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 18 21:08:20.961509 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:20.961478 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 18 21:08:21.237295 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.237257 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-client-ca-bundle\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.237516 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.237336 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.237516 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.237372 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-tls\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.237516 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.237441 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-client-certs\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.237516 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.237481 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-metrics-server-audit-profiles\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.238152 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.238124 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.238406 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.238379 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-metrics-server-audit-profiles\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.239885 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.239852 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-client-certs\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.239885 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.239871 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-secret-metrics-server-tls\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.240020 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.239927 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31cbff63-8b3a-420d-a9b3-9310f05cbcd3-client-ca-bundle\") pod \"metrics-server-89cb5fbc4-svcnv\" (UID: \"31cbff63-8b3a-420d-a9b3-9310f05cbcd3\") " pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.408407 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.408364 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:21.682071 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:21.682032 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-89cb5fbc4-svcnv"] Sep 18 21:08:21.685166 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:08:21.685131 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31cbff63_8b3a_420d_a9b3_9310f05cbcd3.slice/crio-1335783e792eeaa3bbaa39263753d6e76b4ac7bfb7db6bf3b865409bfbbaeca3 WatchSource:0}: Error finding container 1335783e792eeaa3bbaa39263753d6e76b4ac7bfb7db6bf3b865409bfbbaeca3: Status 404 returned error can't find the container with id 1335783e792eeaa3bbaa39263753d6e76b4ac7bfb7db6bf3b865409bfbbaeca3 Sep 18 21:08:22.507507 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:22.507468 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" event={"ID":"31cbff63-8b3a-420d-a9b3-9310f05cbcd3","Type":"ContainerStarted","Data":"1335783e792eeaa3bbaa39263753d6e76b4ac7bfb7db6bf3b865409bfbbaeca3"} Sep 18 21:08:23.514535 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:23.514498 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" event={"ID":"31cbff63-8b3a-420d-a9b3-9310f05cbcd3","Type":"ContainerStarted","Data":"2612a08d82cb5a03d60d52606a331eb19828692461121cb958328bc4280da114"} Sep 18 21:08:23.545033 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:23.544975 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" podStartSLOduration=3.277014922 podStartE2EDuration="4.544957854s" totalImagesPullingTime="1.267942932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:08:23.544479838 +0000 UTC m=+146.193077442" watchObservedRunningTime="2026-09-18 21:08:23.544957854 +0000 UTC m=+146.193555445" Sep 18 21:08:28.060587 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.060549 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b86549d88-bbwrm"] Sep 18 21:08:28.061102 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:28.060857 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" podUID="4251e8c2-55f8-488c-96f1-89cb5ffc93f0" Sep 18 21:08:28.531120 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.531091 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:08:28.540593 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.540571 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:08:28.595877 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.595840 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qprmn\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-kube-api-access-qprmn\") pod \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " Sep 18 21:08:28.596056 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.595890 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-installation-pull-secrets\") pod \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " Sep 18 21:08:28.596056 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.595914 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-certificates\") pod \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " Sep 18 21:08:28.596056 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.595945 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-ca-trust-extracted\") pod \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " Sep 18 21:08:28.596056 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.595978 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-trusted-ca\") pod \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " Sep 18 21:08:28.596056 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.595995 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-bound-sa-token\") pod \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " Sep 18 21:08:28.596056 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.596022 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-image-registry-private-configuration\") pod \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\" (UID: \"4251e8c2-55f8-488c-96f1-89cb5ffc93f0\") " Sep 18 21:08:28.596390 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.596365 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4251e8c2-55f8-488c-96f1-89cb5ffc93f0" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:08:28.596451 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.596387 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4251e8c2-55f8-488c-96f1-89cb5ffc93f0" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 21:08:28.596561 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.596534 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4251e8c2-55f8-488c-96f1-89cb5ffc93f0" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 21:08:28.598306 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.598281 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4251e8c2-55f8-488c-96f1-89cb5ffc93f0" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:08:28.598424 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.598339 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4251e8c2-55f8-488c-96f1-89cb5ffc93f0" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 21:08:28.598505 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.598484 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-kube-api-access-qprmn" (OuterVolumeSpecName: "kube-api-access-qprmn") pod "4251e8c2-55f8-488c-96f1-89cb5ffc93f0" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0"). InnerVolumeSpecName "kube-api-access-qprmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:08:28.598585 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.598569 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4251e8c2-55f8-488c-96f1-89cb5ffc93f0" (UID: "4251e8c2-55f8-488c-96f1-89cb5ffc93f0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 21:08:28.697241 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.697201 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-ca-trust-extracted\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:08:28.697241 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.697235 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-trusted-ca\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:08:28.697241 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.697244 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-bound-sa-token\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:08:28.697241 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.697252 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-image-registry-private-configuration\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:08:28.697514 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.697262 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qprmn\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-kube-api-access-qprmn\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:08:28.697514 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.697271 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-installation-pull-secrets\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:08:28.697514 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:28.697278 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-certificates\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:08:29.534728 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:29.534697 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b86549d88-bbwrm" Sep 18 21:08:29.583901 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:29.583870 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b86549d88-bbwrm"] Sep 18 21:08:29.600100 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:29.600072 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b86549d88-bbwrm"] Sep 18 21:08:29.705201 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:29.705155 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4251e8c2-55f8-488c-96f1-89cb5ffc93f0-registry-tls\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:08:29.858798 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:29.858765 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4251e8c2-55f8-488c-96f1-89cb5ffc93f0" path="/var/lib/kubelet/pods/4251e8c2-55f8-488c-96f1-89cb5ffc93f0/volumes" Sep 18 21:08:35.370986 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:08:35.370934 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tsxlb" podUID="2321d814-a682-4e63-bc8b-c174d6cea7bc" Sep 18 21:08:35.553289 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:35.553258 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tsxlb" Sep 18 21:08:40.291391 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:40.291348 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:08:40.293707 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:40.293678 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2321d814-a682-4e63-bc8b-c174d6cea7bc-metrics-tls\") pod \"dns-default-tsxlb\" (UID: \"2321d814-a682-4e63-bc8b-c174d6cea7bc\") " pod="openshift-dns/dns-default-tsxlb" Sep 18 21:08:40.356953 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:40.356918 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qsk7s\"" Sep 18 21:08:40.365214 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:40.365187 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tsxlb" Sep 18 21:08:40.637061 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:40.637028 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tsxlb"] Sep 18 21:08:40.640511 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:08:40.640482 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2321d814_a682_4e63_bc8b_c174d6cea7bc.slice/crio-f67c62d1733ecdd976a9a08d644ccd264a6d50d70cf7b388e8d87ae97271ddd8 WatchSource:0}: Error finding container f67c62d1733ecdd976a9a08d644ccd264a6d50d70cf7b388e8d87ae97271ddd8: Status 404 returned error can't find the container with id f67c62d1733ecdd976a9a08d644ccd264a6d50d70cf7b388e8d87ae97271ddd8 Sep 18 21:08:41.409264 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:41.409231 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:41.409797 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:41.409303 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:08:41.575169 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:41.575124 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tsxlb" event={"ID":"2321d814-a682-4e63-bc8b-c174d6cea7bc","Type":"ContainerStarted","Data":"f67c62d1733ecdd976a9a08d644ccd264a6d50d70cf7b388e8d87ae97271ddd8"} Sep 18 21:08:42.581175 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:42.581135 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tsxlb" event={"ID":"2321d814-a682-4e63-bc8b-c174d6cea7bc","Type":"ContainerStarted","Data":"200a4a39ab343f65b417627c04a6edd42d4e4a87dfe886bc86bd281f3adc2dbd"} Sep 18 21:08:42.581569 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:42.581197 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tsxlb" event={"ID":"2321d814-a682-4e63-bc8b-c174d6cea7bc","Type":"ContainerStarted","Data":"050d0c07fc06821f5f6fc7665aa975240c01b3436f7aa8d701ce8774ca3ea049"} Sep 18 21:08:42.581569 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:42.581249 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tsxlb" Sep 18 21:08:42.602607 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:42.602558 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tsxlb" podStartSLOduration=129.162132055 podStartE2EDuration="2m10.602540422s" totalImagesPullingTime="1.440408367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:08:42.601887119 +0000 UTC m=+165.250484709" watchObservedRunningTime="2026-09-18 21:08:42.602540422 +0000 UTC m=+165.251138013" Sep 18 21:08:52.587085 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:08:52.587012 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tsxlb" Sep 18 21:09:01.414584 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:09:01.414553 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:09:01.418508 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:09:01.418486 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-89cb5fbc4-svcnv" Sep 18 21:09:10.672372 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:09:10.672334 2557 generic.go:356] "Generic (PLEG): container finished" podID="e4021bfd-62e5-4e7c-8669-fad4ecdd0274" containerID="a8fc760d004beda2197669e7f7622088a99fba1a744538f2c47fb0692e50d264" exitCode=0 Sep 18 21:09:10.672372 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:09:10.672352 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" event={"ID":"e4021bfd-62e5-4e7c-8669-fad4ecdd0274","Type":"ContainerDied","Data":"a8fc760d004beda2197669e7f7622088a99fba1a744538f2c47fb0692e50d264"} Sep 18 21:09:10.672808 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:09:10.672714 2557 scope.go:117] "RemoveContainer" containerID="a8fc760d004beda2197669e7f7622088a99fba1a744538f2c47fb0692e50d264" Sep 18 21:09:11.677788 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:09:11.677755 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2fkqq" event={"ID":"e4021bfd-62e5-4e7c-8669-fad4ecdd0274","Type":"ContainerStarted","Data":"ee1eb3c620e5f38e9e3c50cbb3629ac4fc5e4caf25f2c1ad9f370930d3c40fae"} Sep 18 21:10:57.738058 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:10:57.738030 2557 kubelet.go:1648] "Image garbage collection succeeded" Sep 18 21:11:40.637535 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.637500 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk"] Sep 18 21:11:40.640502 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.640485 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.643428 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.643410 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 18 21:11:40.643825 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.643810 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hcpv9\"" Sep 18 21:11:40.644527 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.644511 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 18 21:11:40.651791 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.651764 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk"] Sep 18 21:11:40.695463 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.695420 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.695632 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.695474 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l5gj\" (UniqueName: \"kubernetes.io/projected/02342e7d-ca44-4512-9b86-fc88a560bc4d-kube-api-access-7l5gj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.695632 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.695545 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.796889 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.796853 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l5gj\" (UniqueName: \"kubernetes.io/projected/02342e7d-ca44-4512-9b86-fc88a560bc4d-kube-api-access-7l5gj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.797049 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.796910 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.797111 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.797086 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.797284 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.797269 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.797421 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.797404 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.804987 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.804968 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l5gj\" (UniqueName: \"kubernetes.io/projected/02342e7d-ca44-4512-9b86-fc88a560bc4d-kube-api-access-7l5gj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:40.949301 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:40.949208 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:41.220667 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:41.220483 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk"] Sep 18 21:11:41.223669 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:11:41.223638 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02342e7d_ca44_4512_9b86_fc88a560bc4d.slice/crio-f5b718a5c4bf5ba8ccbe8f8f44bb4ed3ee432cac0223c692cd3a08e54522f723 WatchSource:0}: Error finding container f5b718a5c4bf5ba8ccbe8f8f44bb4ed3ee432cac0223c692cd3a08e54522f723: Status 404 returned error can't find the container with id f5b718a5c4bf5ba8ccbe8f8f44bb4ed3ee432cac0223c692cd3a08e54522f723 Sep 18 21:11:41.225579 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:41.225562 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 21:11:42.171981 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:42.171946 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" event={"ID":"02342e7d-ca44-4512-9b86-fc88a560bc4d","Type":"ContainerStarted","Data":"f5b718a5c4bf5ba8ccbe8f8f44bb4ed3ee432cac0223c692cd3a08e54522f723"} Sep 18 21:11:48.201435 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:48.201397 2557 generic.go:356] "Generic (PLEG): container finished" podID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerID="04dc1b7a803bcb5bb42da0aee003e2a999577f680e782cc5c7c6206ee5069831" exitCode=0 Sep 18 21:11:48.201435 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:48.201438 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" event={"ID":"02342e7d-ca44-4512-9b86-fc88a560bc4d","Type":"ContainerDied","Data":"04dc1b7a803bcb5bb42da0aee003e2a999577f680e782cc5c7c6206ee5069831"} Sep 18 21:11:51.215501 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:51.215461 2557 generic.go:356] "Generic (PLEG): container finished" podID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerID="275e09bb3d389114f8f1d574fcf89767d6aee7b930806ebaa0af5aca1c6e1eb1" exitCode=0 Sep 18 21:11:51.215852 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:51.215545 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" event={"ID":"02342e7d-ca44-4512-9b86-fc88a560bc4d","Type":"ContainerDied","Data":"275e09bb3d389114f8f1d574fcf89767d6aee7b930806ebaa0af5aca1c6e1eb1"} Sep 18 21:11:58.242868 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:58.242831 2557 generic.go:356] "Generic (PLEG): container finished" podID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerID="3b758aa02998112c178bffa12e0190256aae5221e7980b37d195043ce72f7dbf" exitCode=0 Sep 18 21:11:58.243360 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:58.242892 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" event={"ID":"02342e7d-ca44-4512-9b86-fc88a560bc4d","Type":"ContainerDied","Data":"3b758aa02998112c178bffa12e0190256aae5221e7980b37d195043ce72f7dbf"} Sep 18 21:11:59.367305 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.367280 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:11:59.457254 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.457221 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l5gj\" (UniqueName: \"kubernetes.io/projected/02342e7d-ca44-4512-9b86-fc88a560bc4d-kube-api-access-7l5gj\") pod \"02342e7d-ca44-4512-9b86-fc88a560bc4d\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " Sep 18 21:11:59.457254 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.457260 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-bundle\") pod \"02342e7d-ca44-4512-9b86-fc88a560bc4d\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " Sep 18 21:11:59.457545 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.457283 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-util\") pod \"02342e7d-ca44-4512-9b86-fc88a560bc4d\" (UID: \"02342e7d-ca44-4512-9b86-fc88a560bc4d\") " Sep 18 21:11:59.457940 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.457913 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-bundle" (OuterVolumeSpecName: "bundle") pod "02342e7d-ca44-4512-9b86-fc88a560bc4d" (UID: "02342e7d-ca44-4512-9b86-fc88a560bc4d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:11:59.459521 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.459498 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02342e7d-ca44-4512-9b86-fc88a560bc4d-kube-api-access-7l5gj" (OuterVolumeSpecName: "kube-api-access-7l5gj") pod "02342e7d-ca44-4512-9b86-fc88a560bc4d" (UID: "02342e7d-ca44-4512-9b86-fc88a560bc4d"). InnerVolumeSpecName "kube-api-access-7l5gj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:11:59.462984 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.462958 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-util" (OuterVolumeSpecName: "util") pod "02342e7d-ca44-4512-9b86-fc88a560bc4d" (UID: "02342e7d-ca44-4512-9b86-fc88a560bc4d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:11:59.558392 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.558288 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l5gj\" (UniqueName: \"kubernetes.io/projected/02342e7d-ca44-4512-9b86-fc88a560bc4d-kube-api-access-7l5gj\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:11:59.558392 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.558347 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-bundle\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:11:59.558392 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:11:59.558356 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02342e7d-ca44-4512-9b86-fc88a560bc4d-util\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:00.251555 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:00.251468 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" event={"ID":"02342e7d-ca44-4512-9b86-fc88a560bc4d","Type":"ContainerDied","Data":"f5b718a5c4bf5ba8ccbe8f8f44bb4ed3ee432cac0223c692cd3a08e54522f723"} Sep 18 21:12:00.251555 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:00.251506 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409v5zk" Sep 18 21:12:00.251555 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:00.251518 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5b718a5c4bf5ba8ccbe8f8f44bb4ed3ee432cac0223c692cd3a08e54522f723" Sep 18 21:12:03.844507 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.844475 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4"] Sep 18 21:12:03.844981 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.844803 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerName="extract" Sep 18 21:12:03.844981 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.844821 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerName="extract" Sep 18 21:12:03.844981 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.844834 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerName="pull" Sep 18 21:12:03.844981 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.844843 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerName="pull" Sep 18 21:12:03.844981 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.844869 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerName="util" Sep 18 21:12:03.844981 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.844877 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerName="util" Sep 18 21:12:03.844981 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.844929 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="02342e7d-ca44-4512-9b86-fc88a560bc4d" containerName="extract" Sep 18 21:12:03.848285 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.848263 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" Sep 18 21:12:03.851526 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.851497 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 18 21:12:03.851804 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.851786 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 18 21:12:03.852125 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.852105 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-h5f56\"" Sep 18 21:12:03.865475 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.865445 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4"] Sep 18 21:12:03.990610 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.990572 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s7tq\" (UniqueName: \"kubernetes.io/projected/b92ec4ca-85dd-4966-8a23-af6cf3e338ea-kube-api-access-7s7tq\") pod \"cert-manager-operator-controller-manager-55c444bf86-jdrz4\" (UID: \"b92ec4ca-85dd-4966-8a23-af6cf3e338ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" Sep 18 21:12:03.990793 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:03.990642 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b92ec4ca-85dd-4966-8a23-af6cf3e338ea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jdrz4\" (UID: \"b92ec4ca-85dd-4966-8a23-af6cf3e338ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" Sep 18 21:12:04.091356 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:04.091302 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b92ec4ca-85dd-4966-8a23-af6cf3e338ea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jdrz4\" (UID: \"b92ec4ca-85dd-4966-8a23-af6cf3e338ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" Sep 18 21:12:04.091524 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:04.091366 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s7tq\" (UniqueName: \"kubernetes.io/projected/b92ec4ca-85dd-4966-8a23-af6cf3e338ea-kube-api-access-7s7tq\") pod \"cert-manager-operator-controller-manager-55c444bf86-jdrz4\" (UID: \"b92ec4ca-85dd-4966-8a23-af6cf3e338ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" Sep 18 21:12:04.091804 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:04.091784 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b92ec4ca-85dd-4966-8a23-af6cf3e338ea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jdrz4\" (UID: \"b92ec4ca-85dd-4966-8a23-af6cf3e338ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" Sep 18 21:12:04.108006 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:04.107970 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s7tq\" (UniqueName: \"kubernetes.io/projected/b92ec4ca-85dd-4966-8a23-af6cf3e338ea-kube-api-access-7s7tq\") pod \"cert-manager-operator-controller-manager-55c444bf86-jdrz4\" (UID: \"b92ec4ca-85dd-4966-8a23-af6cf3e338ea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" Sep 18 21:12:04.157569 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:04.157528 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" Sep 18 21:12:04.436224 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:04.436198 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4"] Sep 18 21:12:04.438230 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:12:04.438205 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92ec4ca_85dd_4966_8a23_af6cf3e338ea.slice/crio-303a2ffec5a78a3309d28b248a9060d313fad311d05bbd1d09804907d62a6eaa WatchSource:0}: Error finding container 303a2ffec5a78a3309d28b248a9060d313fad311d05bbd1d09804907d62a6eaa: Status 404 returned error can't find the container with id 303a2ffec5a78a3309d28b248a9060d313fad311d05bbd1d09804907d62a6eaa Sep 18 21:12:05.270582 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:05.270536 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" event={"ID":"b92ec4ca-85dd-4966-8a23-af6cf3e338ea","Type":"ContainerStarted","Data":"303a2ffec5a78a3309d28b248a9060d313fad311d05bbd1d09804907d62a6eaa"} Sep 18 21:12:06.276492 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:06.276446 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" event={"ID":"b92ec4ca-85dd-4966-8a23-af6cf3e338ea","Type":"ContainerStarted","Data":"3b53740ee548198d1de33aeb2a1f5da745aeedda2c20301f541f91ea0cdefddf"} Sep 18 21:12:06.306414 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:06.306349 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jdrz4" podStartSLOduration=1.6378742050000001 podStartE2EDuration="3.306330374s" totalImagesPullingTime="1.668456169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:12:06.304585276 +0000 UTC m=+368.953182870" watchObservedRunningTime="2026-09-18 21:12:06.306330374 +0000 UTC m=+368.954927962" Sep 18 21:12:07.257348 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.257296 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rghkc"] Sep 18 21:12:07.260540 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.260522 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:07.263059 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.263033 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jcmkq\"" Sep 18 21:12:07.264087 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.264067 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 18 21:12:07.264194 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.264118 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 18 21:12:07.274462 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.274437 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rghkc"] Sep 18 21:12:07.418474 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.418434 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/821db6d7-041b-42f8-adc0-28d315b08c73-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rghkc\" (UID: \"821db6d7-041b-42f8-adc0-28d315b08c73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:07.418959 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.418517 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95m9q\" (UniqueName: \"kubernetes.io/projected/821db6d7-041b-42f8-adc0-28d315b08c73-kube-api-access-95m9q\") pod \"cert-manager-webhook-6486ffd997-rghkc\" (UID: \"821db6d7-041b-42f8-adc0-28d315b08c73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:07.519949 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.519849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/821db6d7-041b-42f8-adc0-28d315b08c73-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rghkc\" (UID: \"821db6d7-041b-42f8-adc0-28d315b08c73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:07.519949 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.519929 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95m9q\" (UniqueName: \"kubernetes.io/projected/821db6d7-041b-42f8-adc0-28d315b08c73-kube-api-access-95m9q\") pod \"cert-manager-webhook-6486ffd997-rghkc\" (UID: \"821db6d7-041b-42f8-adc0-28d315b08c73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:07.537620 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.537591 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/821db6d7-041b-42f8-adc0-28d315b08c73-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rghkc\" (UID: \"821db6d7-041b-42f8-adc0-28d315b08c73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:07.537879 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.537858 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95m9q\" (UniqueName: \"kubernetes.io/projected/821db6d7-041b-42f8-adc0-28d315b08c73-kube-api-access-95m9q\") pod \"cert-manager-webhook-6486ffd997-rghkc\" (UID: \"821db6d7-041b-42f8-adc0-28d315b08c73\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:07.569569 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.569537 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:07.793826 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.793743 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct"] Sep 18 21:12:07.798022 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.798004 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:07.804131 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.804107 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 18 21:12:07.804288 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.804183 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 18 21:12:07.805128 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.805107 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hcpv9\"" Sep 18 21:12:07.833027 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.832997 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct"] Sep 18 21:12:07.861442 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.861413 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rghkc"] Sep 18 21:12:07.865041 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:12:07.865020 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod821db6d7_041b_42f8_adc0_28d315b08c73.slice/crio-890cbe6017baa7c49034f78652cdeb4b01dd221a917d649f2d594002d1c33da5 WatchSource:0}: Error finding container 890cbe6017baa7c49034f78652cdeb4b01dd221a917d649f2d594002d1c33da5: Status 404 returned error can't find the container with id 890cbe6017baa7c49034f78652cdeb4b01dd221a917d649f2d594002d1c33da5 Sep 18 21:12:07.923376 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.923338 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:07.923376 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.923377 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:07.923585 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:07.923497 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26mv4\" (UniqueName: \"kubernetes.io/projected/cab5102d-3544-40ff-9cd9-b3f8b0623058-kube-api-access-26mv4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:08.024904 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:08.024854 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26mv4\" (UniqueName: \"kubernetes.io/projected/cab5102d-3544-40ff-9cd9-b3f8b0623058-kube-api-access-26mv4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:08.024904 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:08.024916 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:08.025168 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:08.024940 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:08.025309 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:08.025293 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:08.025398 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:08.025351 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:08.045826 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:08.045763 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26mv4\" (UniqueName: \"kubernetes.io/projected/cab5102d-3544-40ff-9cd9-b3f8b0623058-kube-api-access-26mv4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:08.107382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:08.107350 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:08.284778 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:08.284741 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" event={"ID":"821db6d7-041b-42f8-adc0-28d315b08c73","Type":"ContainerStarted","Data":"890cbe6017baa7c49034f78652cdeb4b01dd221a917d649f2d594002d1c33da5"} Sep 18 21:12:08.418134 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:08.418098 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct"] Sep 18 21:12:08.431434 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:12:08.431403 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcab5102d_3544_40ff_9cd9_b3f8b0623058.slice/crio-72e2d82c9b95c9adb2103b44d36838d53b8a4b73c8c010aaff87f17f3bc653a6 WatchSource:0}: Error finding container 72e2d82c9b95c9adb2103b44d36838d53b8a4b73c8c010aaff87f17f3bc653a6: Status 404 returned error can't find the container with id 72e2d82c9b95c9adb2103b44d36838d53b8a4b73c8c010aaff87f17f3bc653a6 Sep 18 21:12:09.291196 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:09.291159 2557 generic.go:356] "Generic (PLEG): container finished" podID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerID="26b7033783f65f5ad43f7f5518e8bbc83a70071e0b52315b7149d76106d9ff8d" exitCode=0 Sep 18 21:12:09.291406 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:09.291257 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" event={"ID":"cab5102d-3544-40ff-9cd9-b3f8b0623058","Type":"ContainerDied","Data":"26b7033783f65f5ad43f7f5518e8bbc83a70071e0b52315b7149d76106d9ff8d"} Sep 18 21:12:09.291406 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:09.291331 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" event={"ID":"cab5102d-3544-40ff-9cd9-b3f8b0623058","Type":"ContainerStarted","Data":"72e2d82c9b95c9adb2103b44d36838d53b8a4b73c8c010aaff87f17f3bc653a6"} Sep 18 21:12:12.305633 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:12.305596 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" event={"ID":"821db6d7-041b-42f8-adc0-28d315b08c73","Type":"ContainerStarted","Data":"a95c6e4654135470a348c6113a79befeb15d6cb819f42baf9bb884f0cbeb06a0"} Sep 18 21:12:12.306061 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:12.305688 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:12.307565 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:12.307540 2557 generic.go:356] "Generic (PLEG): container finished" podID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerID="67978483aa49c81495809c60b6e71aa915dba2bea6e52dd193d6b93e008b78cc" exitCode=0 Sep 18 21:12:12.307669 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:12.307595 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" event={"ID":"cab5102d-3544-40ff-9cd9-b3f8b0623058","Type":"ContainerDied","Data":"67978483aa49c81495809c60b6e71aa915dba2bea6e52dd193d6b93e008b78cc"} Sep 18 21:12:12.323645 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:12.323598 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" podStartSLOduration=1.383106824 podStartE2EDuration="5.323577073s" totalImagesPullingTime="3.940470249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:12:12.3227589 +0000 UTC m=+374.971356491" watchObservedRunningTime="2026-09-18 21:12:12.323577073 +0000 UTC m=+374.972174668" Sep 18 21:12:13.313218 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:13.313181 2557 generic.go:356] "Generic (PLEG): container finished" podID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerID="392cda25a5691192b3be49f9d67eea414c960df9e188af1bf14f9348f8490d32" exitCode=0 Sep 18 21:12:13.313607 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:13.313263 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" event={"ID":"cab5102d-3544-40ff-9cd9-b3f8b0623058","Type":"ContainerDied","Data":"392cda25a5691192b3be49f9d67eea414c960df9e188af1bf14f9348f8490d32"} Sep 18 21:12:14.439564 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.439541 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:14.576972 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.576884 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-util\") pod \"cab5102d-3544-40ff-9cd9-b3f8b0623058\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " Sep 18 21:12:14.576972 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.576956 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26mv4\" (UniqueName: \"kubernetes.io/projected/cab5102d-3544-40ff-9cd9-b3f8b0623058-kube-api-access-26mv4\") pod \"cab5102d-3544-40ff-9cd9-b3f8b0623058\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " Sep 18 21:12:14.577163 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.576998 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-bundle\") pod \"cab5102d-3544-40ff-9cd9-b3f8b0623058\" (UID: \"cab5102d-3544-40ff-9cd9-b3f8b0623058\") " Sep 18 21:12:14.577410 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.577381 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-bundle" (OuterVolumeSpecName: "bundle") pod "cab5102d-3544-40ff-9cd9-b3f8b0623058" (UID: "cab5102d-3544-40ff-9cd9-b3f8b0623058"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:12:14.579168 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.579145 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cab5102d-3544-40ff-9cd9-b3f8b0623058-kube-api-access-26mv4" (OuterVolumeSpecName: "kube-api-access-26mv4") pod "cab5102d-3544-40ff-9cd9-b3f8b0623058" (UID: "cab5102d-3544-40ff-9cd9-b3f8b0623058"). InnerVolumeSpecName "kube-api-access-26mv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:12:14.581549 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.581525 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-util" (OuterVolumeSpecName: "util") pod "cab5102d-3544-40ff-9cd9-b3f8b0623058" (UID: "cab5102d-3544-40ff-9cd9-b3f8b0623058"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:12:14.677827 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.677770 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26mv4\" (UniqueName: \"kubernetes.io/projected/cab5102d-3544-40ff-9cd9-b3f8b0623058-kube-api-access-26mv4\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:14.677827 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.677819 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-bundle\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:14.677827 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:14.677829 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cab5102d-3544-40ff-9cd9-b3f8b0623058-util\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:15.323303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:15.323269 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" event={"ID":"cab5102d-3544-40ff-9cd9-b3f8b0623058","Type":"ContainerDied","Data":"72e2d82c9b95c9adb2103b44d36838d53b8a4b73c8c010aaff87f17f3bc653a6"} Sep 18 21:12:15.323303 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:15.323286 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvv8ct" Sep 18 21:12:15.323519 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:15.323335 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72e2d82c9b95c9adb2103b44d36838d53b8a4b73c8c010aaff87f17f3bc653a6" Sep 18 21:12:18.315399 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.315370 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rghkc" Sep 18 21:12:18.749346 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.749297 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9pj59"] Sep 18 21:12:18.749639 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.749622 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerName="util" Sep 18 21:12:18.749720 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.749642 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerName="util" Sep 18 21:12:18.749720 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.749655 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerName="pull" Sep 18 21:12:18.749720 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.749664 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerName="pull" Sep 18 21:12:18.749720 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.749674 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerName="extract" Sep 18 21:12:18.749720 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.749682 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerName="extract" Sep 18 21:12:18.749946 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.749758 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cab5102d-3544-40ff-9cd9-b3f8b0623058" containerName="extract" Sep 18 21:12:18.765170 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.765143 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9pj59" Sep 18 21:12:18.768224 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.768202 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9pbmt\"" Sep 18 21:12:18.769647 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.769621 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9pj59"] Sep 18 21:12:18.806295 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.806259 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d64jw\" (UniqueName: \"kubernetes.io/projected/04f822e8-fdb6-4623-8b75-a7f5172221ff-kube-api-access-d64jw\") pod \"cert-manager-55b4d954d5-9pj59\" (UID: \"04f822e8-fdb6-4623-8b75-a7f5172221ff\") " pod="cert-manager/cert-manager-55b4d954d5-9pj59" Sep 18 21:12:18.806295 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.806297 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04f822e8-fdb6-4623-8b75-a7f5172221ff-bound-sa-token\") pod \"cert-manager-55b4d954d5-9pj59\" (UID: \"04f822e8-fdb6-4623-8b75-a7f5172221ff\") " pod="cert-manager/cert-manager-55b4d954d5-9pj59" Sep 18 21:12:18.907149 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.907110 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d64jw\" (UniqueName: \"kubernetes.io/projected/04f822e8-fdb6-4623-8b75-a7f5172221ff-kube-api-access-d64jw\") pod \"cert-manager-55b4d954d5-9pj59\" (UID: \"04f822e8-fdb6-4623-8b75-a7f5172221ff\") " pod="cert-manager/cert-manager-55b4d954d5-9pj59" Sep 18 21:12:18.907149 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.907149 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04f822e8-fdb6-4623-8b75-a7f5172221ff-bound-sa-token\") pod \"cert-manager-55b4d954d5-9pj59\" (UID: \"04f822e8-fdb6-4623-8b75-a7f5172221ff\") " pod="cert-manager/cert-manager-55b4d954d5-9pj59" Sep 18 21:12:18.916255 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.916219 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04f822e8-fdb6-4623-8b75-a7f5172221ff-bound-sa-token\") pod \"cert-manager-55b4d954d5-9pj59\" (UID: \"04f822e8-fdb6-4623-8b75-a7f5172221ff\") " pod="cert-manager/cert-manager-55b4d954d5-9pj59" Sep 18 21:12:18.916401 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:18.916386 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d64jw\" (UniqueName: \"kubernetes.io/projected/04f822e8-fdb6-4623-8b75-a7f5172221ff-kube-api-access-d64jw\") pod \"cert-manager-55b4d954d5-9pj59\" (UID: \"04f822e8-fdb6-4623-8b75-a7f5172221ff\") " pod="cert-manager/cert-manager-55b4d954d5-9pj59" Sep 18 21:12:19.074879 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:19.074796 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9pj59" Sep 18 21:12:19.354403 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:19.354370 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9pj59"] Sep 18 21:12:19.357270 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:12:19.357245 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04f822e8_fdb6_4623_8b75_a7f5172221ff.slice/crio-2385ad048833adc7b2a54a27174b1c17753118f7adab037cde5bb201d0a5d0a0 WatchSource:0}: Error finding container 2385ad048833adc7b2a54a27174b1c17753118f7adab037cde5bb201d0a5d0a0: Status 404 returned error can't find the container with id 2385ad048833adc7b2a54a27174b1c17753118f7adab037cde5bb201d0a5d0a0 Sep 18 21:12:20.348528 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:20.348488 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9pj59" event={"ID":"04f822e8-fdb6-4623-8b75-a7f5172221ff","Type":"ContainerStarted","Data":"3e1e407e2c4700fee692b9296b7026433b6d0ef389473b20f1e6d78d7be1cdda"} Sep 18 21:12:20.348528 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:20.348536 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9pj59" event={"ID":"04f822e8-fdb6-4623-8b75-a7f5172221ff","Type":"ContainerStarted","Data":"2385ad048833adc7b2a54a27174b1c17753118f7adab037cde5bb201d0a5d0a0"} Sep 18 21:12:20.368130 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:20.368060 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9pj59" podStartSLOduration=2.368042138 podStartE2EDuration="2.368042138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:12:20.366766654 +0000 UTC m=+383.015364256" watchObservedRunningTime="2026-09-18 21:12:20.368042138 +0000 UTC m=+383.016639730" Sep 18 21:12:27.681027 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.680993 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t"] Sep 18 21:12:27.688007 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.687986 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.692989 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.692969 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 18 21:12:27.693388 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.693369 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 18 21:12:27.694099 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.694081 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hcpv9\"" Sep 18 21:12:27.700900 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.700875 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t"] Sep 18 21:12:27.772422 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.772386 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.772590 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.772432 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt2xp\" (UniqueName: \"kubernetes.io/projected/713855ed-9057-4e69-b07c-586c3ef2623a-kube-api-access-dt2xp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.772590 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.772533 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.872904 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.872870 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.873086 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.872914 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.873086 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.872951 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt2xp\" (UniqueName: \"kubernetes.io/projected/713855ed-9057-4e69-b07c-586c3ef2623a-kube-api-access-dt2xp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.873326 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.873284 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.873423 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.873339 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.885821 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.885789 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt2xp\" (UniqueName: \"kubernetes.io/projected/713855ed-9057-4e69-b07c-586c3ef2623a-kube-api-access-dt2xp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:27.996652 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:27.996568 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:28.282770 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:28.282742 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t"] Sep 18 21:12:28.283889 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:12:28.283862 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod713855ed_9057_4e69_b07c_586c3ef2623a.slice/crio-3e94bab43bd42e9f6d25ca5b3f11c7affc8de5ce5a72fd6c781f7caa5ae38bd5 WatchSource:0}: Error finding container 3e94bab43bd42e9f6d25ca5b3f11c7affc8de5ce5a72fd6c781f7caa5ae38bd5: Status 404 returned error can't find the container with id 3e94bab43bd42e9f6d25ca5b3f11c7affc8de5ce5a72fd6c781f7caa5ae38bd5 Sep 18 21:12:28.382503 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:28.382469 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" event={"ID":"713855ed-9057-4e69-b07c-586c3ef2623a","Type":"ContainerStarted","Data":"81bad49aa8dbcc980308a9af8b1df9950006ce4331e11158ccd19c4913bf3c39"} Sep 18 21:12:28.382627 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:28.382529 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" event={"ID":"713855ed-9057-4e69-b07c-586c3ef2623a","Type":"ContainerStarted","Data":"3e94bab43bd42e9f6d25ca5b3f11c7affc8de5ce5a72fd6c781f7caa5ae38bd5"} Sep 18 21:12:29.388206 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:29.388168 2557 generic.go:356] "Generic (PLEG): container finished" podID="713855ed-9057-4e69-b07c-586c3ef2623a" containerID="81bad49aa8dbcc980308a9af8b1df9950006ce4331e11158ccd19c4913bf3c39" exitCode=0 Sep 18 21:12:29.388669 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:29.388251 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" event={"ID":"713855ed-9057-4e69-b07c-586c3ef2623a","Type":"ContainerDied","Data":"81bad49aa8dbcc980308a9af8b1df9950006ce4331e11158ccd19c4913bf3c39"} Sep 18 21:12:30.394270 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:30.394236 2557 generic.go:356] "Generic (PLEG): container finished" podID="713855ed-9057-4e69-b07c-586c3ef2623a" containerID="9e4b30f0f4a67c56085dbee9f1f6fc78afb3a4870f51cb19889ceb50c8efdd47" exitCode=0 Sep 18 21:12:30.394689 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:30.394279 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" event={"ID":"713855ed-9057-4e69-b07c-586c3ef2623a","Type":"ContainerDied","Data":"9e4b30f0f4a67c56085dbee9f1f6fc78afb3a4870f51cb19889ceb50c8efdd47"} Sep 18 21:12:31.400856 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:31.400826 2557 generic.go:356] "Generic (PLEG): container finished" podID="713855ed-9057-4e69-b07c-586c3ef2623a" containerID="be2a9610bc3af7b91348accf431c902beac394336688f59dd32d0505666ab3f4" exitCode=0 Sep 18 21:12:31.401293 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:31.400892 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" event={"ID":"713855ed-9057-4e69-b07c-586c3ef2623a","Type":"ContainerDied","Data":"be2a9610bc3af7b91348accf431c902beac394336688f59dd32d0505666ab3f4"} Sep 18 21:12:32.546530 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.546496 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:32.607882 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.607836 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-util\") pod \"713855ed-9057-4e69-b07c-586c3ef2623a\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " Sep 18 21:12:32.607882 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.607883 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-bundle\") pod \"713855ed-9057-4e69-b07c-586c3ef2623a\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " Sep 18 21:12:32.608135 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.607918 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt2xp\" (UniqueName: \"kubernetes.io/projected/713855ed-9057-4e69-b07c-586c3ef2623a-kube-api-access-dt2xp\") pod \"713855ed-9057-4e69-b07c-586c3ef2623a\" (UID: \"713855ed-9057-4e69-b07c-586c3ef2623a\") " Sep 18 21:12:32.608681 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.608658 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-bundle" (OuterVolumeSpecName: "bundle") pod "713855ed-9057-4e69-b07c-586c3ef2623a" (UID: "713855ed-9057-4e69-b07c-586c3ef2623a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:12:32.610327 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.610292 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/713855ed-9057-4e69-b07c-586c3ef2623a-kube-api-access-dt2xp" (OuterVolumeSpecName: "kube-api-access-dt2xp") pod "713855ed-9057-4e69-b07c-586c3ef2623a" (UID: "713855ed-9057-4e69-b07c-586c3ef2623a"). InnerVolumeSpecName "kube-api-access-dt2xp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:12:32.613645 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.613599 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-util" (OuterVolumeSpecName: "util") pod "713855ed-9057-4e69-b07c-586c3ef2623a" (UID: "713855ed-9057-4e69-b07c-586c3ef2623a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:12:32.709018 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.708923 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt2xp\" (UniqueName: \"kubernetes.io/projected/713855ed-9057-4e69-b07c-586c3ef2623a-kube-api-access-dt2xp\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:32.709018 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.708957 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-util\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:32.709018 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:32.708970 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/713855ed-9057-4e69-b07c-586c3ef2623a-bundle\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:33.410717 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:33.410679 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" event={"ID":"713855ed-9057-4e69-b07c-586c3ef2623a","Type":"ContainerDied","Data":"3e94bab43bd42e9f6d25ca5b3f11c7affc8de5ce5a72fd6c781f7caa5ae38bd5"} Sep 18 21:12:33.410717 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:33.410714 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz225t" Sep 18 21:12:33.410918 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:33.410733 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e94bab43bd42e9f6d25ca5b3f11c7affc8de5ce5a72fd6c781f7caa5ae38bd5" Sep 18 21:12:39.274215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.274178 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4"] Sep 18 21:12:39.274636 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.274463 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713855ed-9057-4e69-b07c-586c3ef2623a" containerName="extract" Sep 18 21:12:39.274636 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.274476 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="713855ed-9057-4e69-b07c-586c3ef2623a" containerName="extract" Sep 18 21:12:39.274636 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.274497 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713855ed-9057-4e69-b07c-586c3ef2623a" containerName="util" Sep 18 21:12:39.274636 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.274503 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="713855ed-9057-4e69-b07c-586c3ef2623a" containerName="util" Sep 18 21:12:39.274636 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.274509 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713855ed-9057-4e69-b07c-586c3ef2623a" containerName="pull" Sep 18 21:12:39.274636 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.274514 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="713855ed-9057-4e69-b07c-586c3ef2623a" containerName="pull" Sep 18 21:12:39.274636 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.274556 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="713855ed-9057-4e69-b07c-586c3ef2623a" containerName="extract" Sep 18 21:12:39.277045 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.277028 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.282166 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.282141 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 18 21:12:39.282518 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.282496 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 18 21:12:39.282616 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.282496 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hcpv9\"" Sep 18 21:12:39.301307 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.301278 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4"] Sep 18 21:12:39.360745 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.360705 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6q4g\" (UniqueName: \"kubernetes.io/projected/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-kube-api-access-k6q4g\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.360908 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.360755 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.360908 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.360781 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.461461 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.461431 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6q4g\" (UniqueName: \"kubernetes.io/projected/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-kube-api-access-k6q4g\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.461675 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.461472 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.461675 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.461492 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.461857 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.461840 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.461940 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.461920 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.479176 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.479144 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6q4g\" (UniqueName: \"kubernetes.io/projected/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-kube-api-access-k6q4g\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.585354 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.585250 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:39.861760 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:39.861724 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4"] Sep 18 21:12:39.864622 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:12:39.864601 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36bb2d9d_b9a9_48ee_bf9b_e247a0350b08.slice/crio-8eb21a92991b1b72df136697747f5b334513b8142461454e3803ec22c176f8f9 WatchSource:0}: Error finding container 8eb21a92991b1b72df136697747f5b334513b8142461454e3803ec22c176f8f9: Status 404 returned error can't find the container with id 8eb21a92991b1b72df136697747f5b334513b8142461454e3803ec22c176f8f9 Sep 18 21:12:40.445696 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:40.445655 2557 generic.go:356] "Generic (PLEG): container finished" podID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerID="8ad43131b38cee159c73df8e54c13f82292ae8ea43a3d2f1857b56f1ef54b8ba" exitCode=0 Sep 18 21:12:40.446210 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:40.445716 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" event={"ID":"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08","Type":"ContainerDied","Data":"8ad43131b38cee159c73df8e54c13f82292ae8ea43a3d2f1857b56f1ef54b8ba"} Sep 18 21:12:40.446210 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:40.445787 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" event={"ID":"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08","Type":"ContainerStarted","Data":"8eb21a92991b1b72df136697747f5b334513b8142461454e3803ec22c176f8f9"} Sep 18 21:12:41.031372 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.031334 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp"] Sep 18 21:12:41.033573 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.033556 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.037282 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.037255 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 18 21:12:41.037460 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.037304 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 18 21:12:41.037460 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.037338 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 18 21:12:41.037460 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.037400 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rz5vx\"" Sep 18 21:12:41.037460 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.037435 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 18 21:12:41.056374 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.056287 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp"] Sep 18 21:12:41.075999 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.075969 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de53b195-a3ff-4a12-aa70-a1395c1fc918-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.075999 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.076000 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de53b195-a3ff-4a12-aa70-a1395c1fc918-tmp\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.076189 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.076022 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/de53b195-a3ff-4a12-aa70-a1395c1fc918-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.076189 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.076135 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de53b195-a3ff-4a12-aa70-a1395c1fc918-manifests\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.076189 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.076167 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgsth\" (UniqueName: \"kubernetes.io/projected/de53b195-a3ff-4a12-aa70-a1395c1fc918-kube-api-access-kgsth\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.176684 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.176643 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de53b195-a3ff-4a12-aa70-a1395c1fc918-manifests\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.176684 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.176683 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgsth\" (UniqueName: \"kubernetes.io/projected/de53b195-a3ff-4a12-aa70-a1395c1fc918-kube-api-access-kgsth\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.176930 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.176720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de53b195-a3ff-4a12-aa70-a1395c1fc918-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.176930 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.176737 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de53b195-a3ff-4a12-aa70-a1395c1fc918-tmp\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.176930 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.176757 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/de53b195-a3ff-4a12-aa70-a1395c1fc918-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.177144 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.177115 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de53b195-a3ff-4a12-aa70-a1395c1fc918-manifests\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.179100 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.179067 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de53b195-a3ff-4a12-aa70-a1395c1fc918-tmp\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.179215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.179179 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/de53b195-a3ff-4a12-aa70-a1395c1fc918-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.179215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.179201 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de53b195-a3ff-4a12-aa70-a1395c1fc918-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.188782 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.188760 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgsth\" (UniqueName: \"kubernetes.io/projected/de53b195-a3ff-4a12-aa70-a1395c1fc918-kube-api-access-kgsth\") pod \"opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp\" (UID: \"de53b195-a3ff-4a12-aa70-a1395c1fc918\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.343770 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.343672 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:41.452712 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.452676 2557 generic.go:356] "Generic (PLEG): container finished" podID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerID="38ccb3d5948d5d7f0b429014baff95891bbfa246a5bb696aaef933bebe5e2153" exitCode=0 Sep 18 21:12:41.453078 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.452741 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" event={"ID":"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08","Type":"ContainerDied","Data":"38ccb3d5948d5d7f0b429014baff95891bbfa246a5bb696aaef933bebe5e2153"} Sep 18 21:12:41.635496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:41.635471 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp"] Sep 18 21:12:41.662217 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:12:41.662172 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde53b195_a3ff_4a12_aa70_a1395c1fc918.slice/crio-5c361633c295eb66a03d82a7102c2c11a6f782f2c7dd31cc062689121b77a0ca WatchSource:0}: Error finding container 5c361633c295eb66a03d82a7102c2c11a6f782f2c7dd31cc062689121b77a0ca: Status 404 returned error can't find the container with id 5c361633c295eb66a03d82a7102c2c11a6f782f2c7dd31cc062689121b77a0ca Sep 18 21:12:42.461859 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:42.461822 2557 generic.go:356] "Generic (PLEG): container finished" podID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerID="b1c6447f854a34bd4dbfebaad9200bba9a38e4b5a6cbb3cb015681bdb2baab01" exitCode=0 Sep 18 21:12:42.462334 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:42.461949 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" event={"ID":"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08","Type":"ContainerDied","Data":"b1c6447f854a34bd4dbfebaad9200bba9a38e4b5a6cbb3cb015681bdb2baab01"} Sep 18 21:12:42.463439 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:42.463411 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" event={"ID":"de53b195-a3ff-4a12-aa70-a1395c1fc918","Type":"ContainerStarted","Data":"5c361633c295eb66a03d82a7102c2c11a6f782f2c7dd31cc062689121b77a0ca"} Sep 18 21:12:44.549267 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.549243 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:44.608429 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.608397 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6q4g\" (UniqueName: \"kubernetes.io/projected/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-kube-api-access-k6q4g\") pod \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " Sep 18 21:12:44.608537 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.608504 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-bundle\") pod \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " Sep 18 21:12:44.608608 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.608550 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-util\") pod \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\" (UID: \"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08\") " Sep 18 21:12:44.609568 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.609536 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-bundle" (OuterVolumeSpecName: "bundle") pod "36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" (UID: "36bb2d9d-b9a9-48ee-bf9b-e247a0350b08"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:12:44.610606 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.610582 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-kube-api-access-k6q4g" (OuterVolumeSpecName: "kube-api-access-k6q4g") pod "36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" (UID: "36bb2d9d-b9a9-48ee-bf9b-e247a0350b08"). InnerVolumeSpecName "kube-api-access-k6q4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:12:44.614557 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.614533 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-util" (OuterVolumeSpecName: "util") pod "36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" (UID: "36bb2d9d-b9a9-48ee-bf9b-e247a0350b08"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:12:44.709968 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.709931 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-bundle\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:44.709968 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.709960 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-util\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:44.710166 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:44.710002 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6q4g\" (UniqueName: \"kubernetes.io/projected/36bb2d9d-b9a9-48ee-bf9b-e247a0350b08-kube-api-access-k6q4g\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:12:45.480330 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:45.480269 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" event={"ID":"de53b195-a3ff-4a12-aa70-a1395c1fc918","Type":"ContainerStarted","Data":"9984377cead06dd7762cf28f827511d7d28addf90b409fad78a95c520ffee608"} Sep 18 21:12:45.482143 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:45.482117 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" event={"ID":"36bb2d9d-b9a9-48ee-bf9b-e247a0350b08","Type":"ContainerDied","Data":"8eb21a92991b1b72df136697747f5b334513b8142461454e3803ec22c176f8f9"} Sep 18 21:12:45.482265 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:45.482146 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0vhmz4" Sep 18 21:12:45.482265 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:45.482156 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8eb21a92991b1b72df136697747f5b334513b8142461454e3803ec22c176f8f9" Sep 18 21:12:46.487934 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:46.487898 2557 generic.go:356] "Generic (PLEG): container finished" podID="de53b195-a3ff-4a12-aa70-a1395c1fc918" containerID="9984377cead06dd7762cf28f827511d7d28addf90b409fad78a95c520ffee608" exitCode=0 Sep 18 21:12:46.488348 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:46.487973 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" event={"ID":"de53b195-a3ff-4a12-aa70-a1395c1fc918","Type":"ContainerDied","Data":"9984377cead06dd7762cf28f827511d7d28addf90b409fad78a95c520ffee608"} Sep 18 21:12:47.493963 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:47.493928 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" event={"ID":"de53b195-a3ff-4a12-aa70-a1395c1fc918","Type":"ContainerStarted","Data":"bbba662f97408dc32e8008f095e7c8d4a0676c9151c31f20e066fa2dc6c86c8d"} Sep 18 21:12:47.494411 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:47.494067 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:58.501229 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:58.501195 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" Sep 18 21:12:58.527391 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:12:58.527333 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp" podStartSLOduration=14.225574072 podStartE2EDuration="18.527297853s" totalImagesPullingTime="3.2298279s" totalInitContainerRuntime="1.071895881s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:12:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:12:47.532511924 +0000 UTC m=+410.181109516" watchObservedRunningTime="2026-09-18 21:12:58.527297853 +0000 UTC m=+421.175895444" Sep 18 21:13:05.973843 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.973803 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j"] Sep 18 21:13:05.974228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.974055 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerName="pull" Sep 18 21:13:05.974228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.974065 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerName="pull" Sep 18 21:13:05.974228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.974072 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerName="extract" Sep 18 21:13:05.974228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.974077 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerName="extract" Sep 18 21:13:05.974228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.974089 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerName="util" Sep 18 21:13:05.974228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.974095 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerName="util" Sep 18 21:13:05.974228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.974146 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="36bb2d9d-b9a9-48ee-bf9b-e247a0350b08" containerName="extract" Sep 18 21:13:05.978932 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.978911 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:05.981563 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.981544 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-x6ccr\"" Sep 18 21:13:05.981741 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.981722 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 18 21:13:05.981835 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.981816 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 18 21:13:05.982686 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.982655 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 18 21:13:05.982790 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.982721 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 18 21:13:05.989583 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:05.989561 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j"] Sep 18 21:13:06.087028 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.086986 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed9db4a-726e-4f14-84e6-c27bf160c765-tmp\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.087215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.087088 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed9db4a-726e-4f14-84e6-c27bf160c765-tls-certs\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.087215 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.087160 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs8tg\" (UniqueName: \"kubernetes.io/projected/2ed9db4a-726e-4f14-84e6-c27bf160c765-kube-api-access-vs8tg\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.188086 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.188042 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed9db4a-726e-4f14-84e6-c27bf160c765-tmp\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.188293 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.188099 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed9db4a-726e-4f14-84e6-c27bf160c765-tls-certs\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.188293 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.188128 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs8tg\" (UniqueName: \"kubernetes.io/projected/2ed9db4a-726e-4f14-84e6-c27bf160c765-kube-api-access-vs8tg\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.188293 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:13:06.188222 2557 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 18 21:13:06.188483 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:13:06.188299 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed9db4a-726e-4f14-84e6-c27bf160c765-tls-certs podName:2ed9db4a-726e-4f14-84e6-c27bf160c765 nodeName:}" failed. No retries permitted until 2026-09-18 21:13:06.688282961 +0000 UTC m=+429.336880529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2ed9db4a-726e-4f14-84e6-c27bf160c765-tls-certs") pod "kube-auth-proxy-bf5bc8c8c-kss4j" (UID: "2ed9db4a-726e-4f14-84e6-c27bf160c765") : secret "kube-auth-proxy-tls" not found Sep 18 21:13:06.190403 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.190378 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed9db4a-726e-4f14-84e6-c27bf160c765-tmp\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.196428 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.196405 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs8tg\" (UniqueName: \"kubernetes.io/projected/2ed9db4a-726e-4f14-84e6-c27bf160c765-kube-api-access-vs8tg\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.692142 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.692103 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed9db4a-726e-4f14-84e6-c27bf160c765-tls-certs\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.694614 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.694586 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed9db4a-726e-4f14-84e6-c27bf160c765-tls-certs\") pod \"kube-auth-proxy-bf5bc8c8c-kss4j\" (UID: \"2ed9db4a-726e-4f14-84e6-c27bf160c765\") " pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:06.889253 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:06.889219 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" Sep 18 21:13:07.169864 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.169837 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j"] Sep 18 21:13:07.172239 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:13:07.172208 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed9db4a_726e_4f14_84e6_c27bf160c765.slice/crio-8f51bad6b329ffceaaa17eadc3830868e9a77b27b71cec07fe0c5a7dc113b8e0 WatchSource:0}: Error finding container 8f51bad6b329ffceaaa17eadc3830868e9a77b27b71cec07fe0c5a7dc113b8e0: Status 404 returned error can't find the container with id 8f51bad6b329ffceaaa17eadc3830868e9a77b27b71cec07fe0c5a7dc113b8e0 Sep 18 21:13:07.556152 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.556071 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn"] Sep 18 21:13:07.560338 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.560300 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.567642 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.567622 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 18 21:13:07.567765 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.567624 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 18 21:13:07.567880 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.567865 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 18 21:13:07.567928 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.567865 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 18 21:13:07.575688 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.575671 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vrp54\"" Sep 18 21:13:07.583024 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.582992 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" event={"ID":"2ed9db4a-726e-4f14-84e6-c27bf160c765","Type":"ContainerStarted","Data":"8f51bad6b329ffceaaa17eadc3830868e9a77b27b71cec07fe0c5a7dc113b8e0"} Sep 18 21:13:07.599004 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.598976 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn"] Sep 18 21:13:07.700715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.700399 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.700715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.700459 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.700715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.700499 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.700715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.700524 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc8g7\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-kube-api-access-mc8g7\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.700715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.700596 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.700715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.700666 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.700715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.700722 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.801411 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.801356 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.801818 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.801705 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.801818 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.801776 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.802194 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.802002 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.802194 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.802064 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.802194 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.802089 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc8g7\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-kube-api-access-mc8g7\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.802194 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.802123 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.802859 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.802813 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.804745 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.804716 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.804935 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.804892 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.805092 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.805068 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.805295 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.805248 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.811462 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.811406 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.811818 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.811795 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc8g7\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-kube-api-access-mc8g7\") pod \"istiod-openshift-gateway-7b7995d5cd-9khdn\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:07.870202 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:07.870171 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:08.165389 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:08.165349 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn"] Sep 18 21:13:08.170196 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:13:08.170162 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadab9967_0d50_4c8c_b13b_99fdf00eeb5f.slice/crio-6a6f9cf891a3c7cccc8f5f6bde9aa7e757aae5826769ed7337eb1ec342550576 WatchSource:0}: Error finding container 6a6f9cf891a3c7cccc8f5f6bde9aa7e757aae5826769ed7337eb1ec342550576: Status 404 returned error can't find the container with id 6a6f9cf891a3c7cccc8f5f6bde9aa7e757aae5826769ed7337eb1ec342550576 Sep 18 21:13:08.591195 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:08.591109 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" event={"ID":"adab9967-0d50-4c8c-b13b-99fdf00eeb5f","Type":"ContainerStarted","Data":"6a6f9cf891a3c7cccc8f5f6bde9aa7e757aae5826769ed7337eb1ec342550576"} Sep 18 21:13:10.603226 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:10.603189 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" event={"ID":"2ed9db4a-726e-4f14-84e6-c27bf160c765","Type":"ContainerStarted","Data":"5e5de139214d09d26e7ba396fdbb748a090d99eeb53ef002867d65bcd9dfd290"} Sep 18 21:13:10.622182 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:10.622117 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-bf5bc8c8c-kss4j" podStartSLOduration=3.070352827 podStartE2EDuration="5.622094564s" totalImagesPullingTime="2.551741737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:13:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:13:10.62067166 +0000 UTC m=+433.269269253" watchObservedRunningTime="2026-09-18 21:13:10.622094564 +0000 UTC m=+433.270692157" Sep 18 21:13:11.563030 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:11.562978 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 18 21:13:11.563111 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:11.563064 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 18 21:13:12.615025 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:12.614986 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" event={"ID":"adab9967-0d50-4c8c-b13b-99fdf00eeb5f","Type":"ContainerStarted","Data":"27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35"} Sep 18 21:13:12.615501 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:12.615134 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:12.617025 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:12.616985 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-9khdn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 18 21:13:12.617149 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:12.617062 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" podUID="adab9967-0d50-4c8c-b13b-99fdf00eeb5f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 18 21:13:12.668760 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:12.668694 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" podStartSLOduration=2.279549709 podStartE2EDuration="5.668670722s" totalImagesPullingTime="3.389121013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:13:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:13:12.666776414 +0000 UTC m=+435.315374029" watchObservedRunningTime="2026-09-18 21:13:12.668670722 +0000 UTC m=+435.317268313" Sep 18 21:13:13.620054 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:13.620027 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:26.079402 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.079304 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-62bxk"] Sep 18 21:13:26.082482 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.082464 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x"] Sep 18 21:13:26.082602 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.082584 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:13:26.085527 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.085504 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:13:26.102331 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.102286 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 18 21:13:26.102454 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.102287 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kg7vz\"" Sep 18 21:13:26.102454 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.102287 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wnpgr\"" Sep 18 21:13:26.110007 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.109986 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 18 21:13:26.114537 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.114514 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x"] Sep 18 21:13:26.120438 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.120412 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-62bxk"] Sep 18 21:13:26.152465 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.152424 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05762543-5954-4a6d-975e-64266f00638b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ns59x\" (UID: \"05762543-5954-4a6d-975e-64266f00638b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:13:26.152643 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.152479 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh7xm\" (UniqueName: \"kubernetes.io/projected/05762543-5954-4a6d-975e-64266f00638b-kube-api-access-qh7xm\") pod \"llmisvc-controller-manager-777c6bd7d5-ns59x\" (UID: \"05762543-5954-4a6d-975e-64266f00638b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:13:26.152643 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.152498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dfa38bb-1d1b-48b0-8fc2-bf11e8702427-cert\") pod \"kserve-controller-manager-d6f4b65b8-62bxk\" (UID: \"3dfa38bb-1d1b-48b0-8fc2-bf11e8702427\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:13:26.152643 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.152539 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk258\" (UniqueName: \"kubernetes.io/projected/3dfa38bb-1d1b-48b0-8fc2-bf11e8702427-kube-api-access-pk258\") pod \"kserve-controller-manager-d6f4b65b8-62bxk\" (UID: \"3dfa38bb-1d1b-48b0-8fc2-bf11e8702427\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:13:26.253382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.253336 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05762543-5954-4a6d-975e-64266f00638b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ns59x\" (UID: \"05762543-5954-4a6d-975e-64266f00638b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:13:26.253570 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.253408 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh7xm\" (UniqueName: \"kubernetes.io/projected/05762543-5954-4a6d-975e-64266f00638b-kube-api-access-qh7xm\") pod \"llmisvc-controller-manager-777c6bd7d5-ns59x\" (UID: \"05762543-5954-4a6d-975e-64266f00638b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:13:26.253570 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.253437 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dfa38bb-1d1b-48b0-8fc2-bf11e8702427-cert\") pod \"kserve-controller-manager-d6f4b65b8-62bxk\" (UID: \"3dfa38bb-1d1b-48b0-8fc2-bf11e8702427\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:13:26.253570 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.253470 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk258\" (UniqueName: \"kubernetes.io/projected/3dfa38bb-1d1b-48b0-8fc2-bf11e8702427-kube-api-access-pk258\") pod \"kserve-controller-manager-d6f4b65b8-62bxk\" (UID: \"3dfa38bb-1d1b-48b0-8fc2-bf11e8702427\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:13:26.256350 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.256304 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05762543-5954-4a6d-975e-64266f00638b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ns59x\" (UID: \"05762543-5954-4a6d-975e-64266f00638b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:13:26.256350 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.256341 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dfa38bb-1d1b-48b0-8fc2-bf11e8702427-cert\") pod \"kserve-controller-manager-d6f4b65b8-62bxk\" (UID: \"3dfa38bb-1d1b-48b0-8fc2-bf11e8702427\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:13:26.268075 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.268050 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh7xm\" (UniqueName: \"kubernetes.io/projected/05762543-5954-4a6d-975e-64266f00638b-kube-api-access-qh7xm\") pod \"llmisvc-controller-manager-777c6bd7d5-ns59x\" (UID: \"05762543-5954-4a6d-975e-64266f00638b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:13:26.275200 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.275175 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk258\" (UniqueName: \"kubernetes.io/projected/3dfa38bb-1d1b-48b0-8fc2-bf11e8702427-kube-api-access-pk258\") pod \"kserve-controller-manager-d6f4b65b8-62bxk\" (UID: \"3dfa38bb-1d1b-48b0-8fc2-bf11e8702427\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:13:26.393391 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.393359 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:13:26.398769 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.398738 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:13:26.724722 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.724699 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-62bxk"] Sep 18 21:13:26.727136 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:13:26.727113 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dfa38bb_1d1b_48b0_8fc2_bf11e8702427.slice/crio-e231cdeb34ed3d870317d5a314ed2f467c647c0203a02c03fe384269929470a7 WatchSource:0}: Error finding container e231cdeb34ed3d870317d5a314ed2f467c647c0203a02c03fe384269929470a7: Status 404 returned error can't find the container with id e231cdeb34ed3d870317d5a314ed2f467c647c0203a02c03fe384269929470a7 Sep 18 21:13:26.740117 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:26.740091 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x"] Sep 18 21:13:26.742873 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:13:26.742849 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod05762543_5954_4a6d_975e_64266f00638b.slice/crio-0a50bb0a3b5c7abf93ad08baa98eb12c4b639d9b191bc622d9ca29461cfc2aec WatchSource:0}: Error finding container 0a50bb0a3b5c7abf93ad08baa98eb12c4b639d9b191bc622d9ca29461cfc2aec: Status 404 returned error can't find the container with id 0a50bb0a3b5c7abf93ad08baa98eb12c4b639d9b191bc622d9ca29461cfc2aec Sep 18 21:13:27.313611 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.313574 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn"] Sep 18 21:13:27.314068 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.313835 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" podUID="adab9967-0d50-4c8c-b13b-99fdf00eeb5f" containerName="discovery" containerID="cri-o://27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35" gracePeriod=30 Sep 18 21:13:27.639584 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.639397 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:27.666681 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.666640 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-token\") pod \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " Sep 18 21:13:27.666864 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.666699 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-cacerts\") pod \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " Sep 18 21:13:27.666864 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.666761 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-local-certs\") pod \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " Sep 18 21:13:27.666864 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.666786 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-kubeconfig\") pod \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " Sep 18 21:13:27.666864 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.666823 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-ca-configmap\") pod \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " Sep 18 21:13:27.666864 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.666850 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-dns-cert\") pod \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " Sep 18 21:13:27.667106 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.666880 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mc8g7\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-kube-api-access-mc8g7\") pod \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\" (UID: \"adab9967-0d50-4c8c-b13b-99fdf00eeb5f\") " Sep 18 21:13:27.668026 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.667973 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "adab9967-0d50-4c8c-b13b-99fdf00eeb5f" (UID: "adab9967-0d50-4c8c-b13b-99fdf00eeb5f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 21:13:27.671213 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.671173 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-local-certs" (OuterVolumeSpecName: "local-certs") pod "adab9967-0d50-4c8c-b13b-99fdf00eeb5f" (UID: "adab9967-0d50-4c8c-b13b-99fdf00eeb5f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:13:27.673133 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.673096 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-kube-api-access-mc8g7" (OuterVolumeSpecName: "kube-api-access-mc8g7") pod "adab9967-0d50-4c8c-b13b-99fdf00eeb5f" (UID: "adab9967-0d50-4c8c-b13b-99fdf00eeb5f"). InnerVolumeSpecName "kube-api-access-mc8g7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:13:27.673530 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.673476 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "adab9967-0d50-4c8c-b13b-99fdf00eeb5f" (UID: "adab9967-0d50-4c8c-b13b-99fdf00eeb5f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 21:13:27.673784 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.673752 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "adab9967-0d50-4c8c-b13b-99fdf00eeb5f" (UID: "adab9967-0d50-4c8c-b13b-99fdf00eeb5f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 21:13:27.674694 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.674660 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-cacerts" (OuterVolumeSpecName: "cacerts") pod "adab9967-0d50-4c8c-b13b-99fdf00eeb5f" (UID: "adab9967-0d50-4c8c-b13b-99fdf00eeb5f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 21:13:27.675214 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.675182 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-token" (OuterVolumeSpecName: "istio-token") pod "adab9967-0d50-4c8c-b13b-99fdf00eeb5f" (UID: "adab9967-0d50-4c8c-b13b-99fdf00eeb5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:13:27.685872 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.685839 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" event={"ID":"3dfa38bb-1d1b-48b0-8fc2-bf11e8702427","Type":"ContainerStarted","Data":"e231cdeb34ed3d870317d5a314ed2f467c647c0203a02c03fe384269929470a7"} Sep 18 21:13:27.687433 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.687407 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" event={"ID":"05762543-5954-4a6d-975e-64266f00638b","Type":"ContainerStarted","Data":"0a50bb0a3b5c7abf93ad08baa98eb12c4b639d9b191bc622d9ca29461cfc2aec"} Sep 18 21:13:27.689624 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.689496 2557 generic.go:356] "Generic (PLEG): container finished" podID="adab9967-0d50-4c8c-b13b-99fdf00eeb5f" containerID="27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35" exitCode=0 Sep 18 21:13:27.689624 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.689572 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" event={"ID":"adab9967-0d50-4c8c-b13b-99fdf00eeb5f","Type":"ContainerDied","Data":"27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35"} Sep 18 21:13:27.689624 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.689619 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" event={"ID":"adab9967-0d50-4c8c-b13b-99fdf00eeb5f","Type":"ContainerDied","Data":"6a6f9cf891a3c7cccc8f5f6bde9aa7e757aae5826769ed7337eb1ec342550576"} Sep 18 21:13:27.690437 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.689646 2557 scope.go:117] "RemoveContainer" containerID="27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35" Sep 18 21:13:27.690437 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.689806 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn" Sep 18 21:13:27.730657 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.730616 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn"] Sep 18 21:13:27.740192 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.740160 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9khdn"] Sep 18 21:13:27.767640 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.767606 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-token\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:13:27.767640 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.767639 2557 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-cacerts\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:13:27.767888 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.767654 2557 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-local-certs\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:13:27.767888 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.767666 2557 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-kubeconfig\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:13:27.767888 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.767680 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-ca-configmap\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:13:27.767888 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.767698 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-istio-csr-dns-cert\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:13:27.767888 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.767711 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc8g7\" (UniqueName: \"kubernetes.io/projected/adab9967-0d50-4c8c-b13b-99fdf00eeb5f-kube-api-access-mc8g7\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:13:27.846552 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.846526 2557 scope.go:117] "RemoveContainer" containerID="27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35" Sep 18 21:13:27.847093 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:13:27.847024 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35\": container with ID starting with 27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35 not found: ID does not exist" containerID="27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35" Sep 18 21:13:27.847224 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.847070 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35"} err="failed to get container status \"27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35\": rpc error: code = NotFound desc = could not find container \"27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35\": container with ID starting with 27c8f99ad401ab59211a401e14f95fd9b1c479e573b45dd1d851e0a88031fc35 not found: ID does not exist" Sep 18 21:13:27.863382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:27.863273 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adab9967-0d50-4c8c-b13b-99fdf00eeb5f" path="/var/lib/kubelet/pods/adab9967-0d50-4c8c-b13b-99fdf00eeb5f/volumes" Sep 18 21:13:29.704380 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:29.704348 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" event={"ID":"05762543-5954-4a6d-975e-64266f00638b","Type":"ContainerStarted","Data":"2a342dcab88087a0affea7f04730546bff81d0775dbc4c6b68c1098df85c1108"} Sep 18 21:13:29.704777 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:29.704444 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:13:29.711123 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:29.711088 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" event={"ID":"3dfa38bb-1d1b-48b0-8fc2-bf11e8702427","Type":"ContainerStarted","Data":"79dcbb8eb5ccf8907ca96567efd4ae7cc223e10d0bcd82c2f9aa855660b36cc3"} Sep 18 21:13:29.711288 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:29.711260 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:13:29.734665 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:29.734608 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" podStartSLOduration=1.8812593949999998 podStartE2EDuration="4.734591083s" totalImagesPullingTime="2.853331688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:13:29.733399466 +0000 UTC m=+452.381997059" watchObservedRunningTime="2026-09-18 21:13:29.734591083 +0000 UTC m=+452.383188674" Sep 18 21:13:29.811030 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:13:29.810976 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" podStartSLOduration=1.945637192 podStartE2EDuration="4.810960485s" totalImagesPullingTime="2.865323293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:13:29.810523002 +0000 UTC m=+452.459120595" watchObservedRunningTime="2026-09-18 21:13:29.810960485 +0000 UTC m=+452.459558077" Sep 18 21:14:00.717013 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:00.716982 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ns59x" Sep 18 21:14:00.720595 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:00.720575 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-62bxk" Sep 18 21:14:19.796954 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.796911 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw"] Sep 18 21:14:19.797570 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.797330 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adab9967-0d50-4c8c-b13b-99fdf00eeb5f" containerName="discovery" Sep 18 21:14:19.797570 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.797350 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="adab9967-0d50-4c8c-b13b-99fdf00eeb5f" containerName="discovery" Sep 18 21:14:19.797570 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.797443 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="adab9967-0d50-4c8c-b13b-99fdf00eeb5f" containerName="discovery" Sep 18 21:14:19.801989 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.801960 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:19.804851 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.804832 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 18 21:14:19.805302 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.805288 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hcpv9\"" Sep 18 21:14:19.808500 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.808291 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 18 21:14:19.812545 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.812525 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw"] Sep 18 21:14:19.872079 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.872045 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:19.872246 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.872091 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spnlw\" (UniqueName: \"kubernetes.io/projected/8ff77b09-4860-469a-b6a0-5cc02d43ab67-kube-api-access-spnlw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:19.872246 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.872183 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:19.928417 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.928378 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44"] Sep 18 21:14:19.930525 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.930508 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:19.956380 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.956351 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44"] Sep 18 21:14:19.973416 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.973386 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:19.973570 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.973426 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:19.973570 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.973459 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spnlw\" (UniqueName: \"kubernetes.io/projected/8ff77b09-4860-469a-b6a0-5cc02d43ab67-kube-api-access-spnlw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:19.973907 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.973882 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:19.974022 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.973949 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:19.987713 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:19.987683 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spnlw\" (UniqueName: \"kubernetes.io/projected/8ff77b09-4860-469a-b6a0-5cc02d43ab67-kube-api-access-spnlw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:20.074549 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.074449 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.074549 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.074508 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.074742 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.074587 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpxkg\" (UniqueName: \"kubernetes.io/projected/dc8c6381-dd07-4db2-903b-2f848431ec95-kube-api-access-gpxkg\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.082200 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.082164 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47"] Sep 18 21:14:20.084642 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.084627 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.111168 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.111137 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:20.114629 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.114596 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47"] Sep 18 21:14:20.176134 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.176097 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.176272 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.176150 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qx2l\" (UniqueName: \"kubernetes.io/projected/c943b701-413f-44d5-808c-61b9aa4f184d-kube-api-access-9qx2l\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.176363 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.176265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.176363 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.176330 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.176477 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.176369 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.176477 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.176402 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpxkg\" (UniqueName: \"kubernetes.io/projected/dc8c6381-dd07-4db2-903b-2f848431ec95-kube-api-access-gpxkg\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.176741 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.176721 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.176916 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.176902 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.192301 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.192263 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpxkg\" (UniqueName: \"kubernetes.io/projected/dc8c6381-dd07-4db2-903b-2f848431ec95-kube-api-access-gpxkg\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.201041 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.201013 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc"] Sep 18 21:14:20.207706 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.207676 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.239774 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.239741 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc"] Sep 18 21:14:20.239968 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.239822 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:20.277742 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.277704 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.277941 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.277777 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.277941 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.277835 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.277941 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.277886 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.277941 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.277915 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qx2l\" (UniqueName: \"kubernetes.io/projected/c943b701-413f-44d5-808c-61b9aa4f184d-kube-api-access-9qx2l\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.278213 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.277955 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4vsv\" (UniqueName: \"kubernetes.io/projected/ad86d541-9ac9-4963-92ae-45d57eb63675-kube-api-access-g4vsv\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.278544 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.278517 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.278725 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.278616 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.289211 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.289181 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qx2l\" (UniqueName: \"kubernetes.io/projected/c943b701-413f-44d5-808c-61b9aa4f184d-kube-api-access-9qx2l\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.378900 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.378864 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4vsv\" (UniqueName: \"kubernetes.io/projected/ad86d541-9ac9-4963-92ae-45d57eb63675-kube-api-access-g4vsv\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.379070 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.378908 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.379070 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.378961 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.379349 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.379305 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.379395 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.379363 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.389196 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.389166 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4vsv\" (UniqueName: \"kubernetes.io/projected/ad86d541-9ac9-4963-92ae-45d57eb63675-kube-api-access-g4vsv\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.394368 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.394284 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:20.394654 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.394634 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw"] Sep 18 21:14:20.397059 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:14:20.397036 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff77b09_4860_469a_b6a0_5cc02d43ab67.slice/crio-6523aafb9971fddf3ad14c71457f2e82d0c699138cd1b5c705d4b9b584c574b4 WatchSource:0}: Error finding container 6523aafb9971fddf3ad14c71457f2e82d0c699138cd1b5c705d4b9b584c574b4: Status 404 returned error can't find the container with id 6523aafb9971fddf3ad14c71457f2e82d0c699138cd1b5c705d4b9b584c574b4 Sep 18 21:14:20.516222 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.516193 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:20.530599 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.530575 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44"] Sep 18 21:14:20.554470 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:14:20.554434 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc8c6381_dd07_4db2_903b_2f848431ec95.slice/crio-51d6d4ca69eb6bc0e9eb5c7c30f0304dd8bb470812125d95cb1914d8efc0383c WatchSource:0}: Error finding container 51d6d4ca69eb6bc0e9eb5c7c30f0304dd8bb470812125d95cb1914d8efc0383c: Status 404 returned error can't find the container with id 51d6d4ca69eb6bc0e9eb5c7c30f0304dd8bb470812125d95cb1914d8efc0383c Sep 18 21:14:20.669714 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.669689 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47"] Sep 18 21:14:20.712103 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:14:20.712068 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc943b701_413f_44d5_808c_61b9aa4f184d.slice/crio-903289666b8f292aa14188aceafc8f84860b30c5a4ebb8aee01b198dad013cd4 WatchSource:0}: Error finding container 903289666b8f292aa14188aceafc8f84860b30c5a4ebb8aee01b198dad013cd4: Status 404 returned error can't find the container with id 903289666b8f292aa14188aceafc8f84860b30c5a4ebb8aee01b198dad013cd4 Sep 18 21:14:20.790217 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.790184 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc"] Sep 18 21:14:20.793172 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:14:20.793137 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad86d541_9ac9_4963_92ae_45d57eb63675.slice/crio-40ecdaf4dc4e335a11dab8a2b042a0449bdbd90fc440814e0391e6a4dbcfeef0 WatchSource:0}: Error finding container 40ecdaf4dc4e335a11dab8a2b042a0449bdbd90fc440814e0391e6a4dbcfeef0: Status 404 returned error can't find the container with id 40ecdaf4dc4e335a11dab8a2b042a0449bdbd90fc440814e0391e6a4dbcfeef0 Sep 18 21:14:20.942517 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.942427 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" event={"ID":"ad86d541-9ac9-4963-92ae-45d57eb63675","Type":"ContainerStarted","Data":"aae7e9c617fc62b7d5139b4ee2fc1c95ba21462eb8ea653dae3688f973ba28bc"} Sep 18 21:14:20.942517 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.942482 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" event={"ID":"ad86d541-9ac9-4963-92ae-45d57eb63675","Type":"ContainerStarted","Data":"40ecdaf4dc4e335a11dab8a2b042a0449bdbd90fc440814e0391e6a4dbcfeef0"} Sep 18 21:14:20.944136 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.944108 2557 generic.go:356] "Generic (PLEG): container finished" podID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerID="52a18057caa0b89ce88ab89bff911912da321cd57cd0d502fc5f85e238e903c0" exitCode=0 Sep 18 21:14:20.944302 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.944180 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" event={"ID":"dc8c6381-dd07-4db2-903b-2f848431ec95","Type":"ContainerDied","Data":"52a18057caa0b89ce88ab89bff911912da321cd57cd0d502fc5f85e238e903c0"} Sep 18 21:14:20.944302 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.944228 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" event={"ID":"dc8c6381-dd07-4db2-903b-2f848431ec95","Type":"ContainerStarted","Data":"51d6d4ca69eb6bc0e9eb5c7c30f0304dd8bb470812125d95cb1914d8efc0383c"} Sep 18 21:14:20.946010 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.945988 2557 generic.go:356] "Generic (PLEG): container finished" podID="c943b701-413f-44d5-808c-61b9aa4f184d" containerID="40e850bed6057c186fc2abe73ebe1b863927dd30479eeec2c993baf519a1f1a1" exitCode=0 Sep 18 21:14:20.946118 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.946073 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" event={"ID":"c943b701-413f-44d5-808c-61b9aa4f184d","Type":"ContainerDied","Data":"40e850bed6057c186fc2abe73ebe1b863927dd30479eeec2c993baf519a1f1a1"} Sep 18 21:14:20.946189 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.946119 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" event={"ID":"c943b701-413f-44d5-808c-61b9aa4f184d","Type":"ContainerStarted","Data":"903289666b8f292aa14188aceafc8f84860b30c5a4ebb8aee01b198dad013cd4"} Sep 18 21:14:20.948003 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.947983 2557 generic.go:356] "Generic (PLEG): container finished" podID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerID="a29f1f86ee9b394de602d857c56e0997469e1fde7406f5a2a6b384ddc9dbb6b8" exitCode=0 Sep 18 21:14:20.948097 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.948037 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" event={"ID":"8ff77b09-4860-469a-b6a0-5cc02d43ab67","Type":"ContainerDied","Data":"a29f1f86ee9b394de602d857c56e0997469e1fde7406f5a2a6b384ddc9dbb6b8"} Sep 18 21:14:20.948097 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:20.948067 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" event={"ID":"8ff77b09-4860-469a-b6a0-5cc02d43ab67","Type":"ContainerStarted","Data":"6523aafb9971fddf3ad14c71457f2e82d0c699138cd1b5c705d4b9b584c574b4"} Sep 18 21:14:21.955556 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:21.955516 2557 generic.go:356] "Generic (PLEG): container finished" podID="c943b701-413f-44d5-808c-61b9aa4f184d" containerID="7cefbde32b23852e4606a409063c79c4add8520965a5b5198ba745f84ed82eef" exitCode=0 Sep 18 21:14:21.955935 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:21.955570 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" event={"ID":"c943b701-413f-44d5-808c-61b9aa4f184d","Type":"ContainerDied","Data":"7cefbde32b23852e4606a409063c79c4add8520965a5b5198ba745f84ed82eef"} Sep 18 21:14:21.957604 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:21.957583 2557 generic.go:356] "Generic (PLEG): container finished" podID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerID="5b2cce290205fb691e035e1360342e574644674f14ce027f47b8b06501b587b6" exitCode=0 Sep 18 21:14:21.957733 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:21.957669 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" event={"ID":"8ff77b09-4860-469a-b6a0-5cc02d43ab67","Type":"ContainerDied","Data":"5b2cce290205fb691e035e1360342e574644674f14ce027f47b8b06501b587b6"} Sep 18 21:14:21.959242 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:21.959223 2557 generic.go:356] "Generic (PLEG): container finished" podID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerID="aae7e9c617fc62b7d5139b4ee2fc1c95ba21462eb8ea653dae3688f973ba28bc" exitCode=0 Sep 18 21:14:21.959365 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:21.959246 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" event={"ID":"ad86d541-9ac9-4963-92ae-45d57eb63675","Type":"ContainerDied","Data":"aae7e9c617fc62b7d5139b4ee2fc1c95ba21462eb8ea653dae3688f973ba28bc"} Sep 18 21:14:21.961279 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:21.961252 2557 generic.go:356] "Generic (PLEG): container finished" podID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerID="30ca11b3d6f149b704eb0fcea791abef169bcf71e7bc33888976d52dfb4d3f4a" exitCode=0 Sep 18 21:14:21.961389 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:21.961283 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" event={"ID":"dc8c6381-dd07-4db2-903b-2f848431ec95","Type":"ContainerDied","Data":"30ca11b3d6f149b704eb0fcea791abef169bcf71e7bc33888976d52dfb4d3f4a"} Sep 18 21:14:22.967718 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:22.967680 2557 generic.go:356] "Generic (PLEG): container finished" podID="c943b701-413f-44d5-808c-61b9aa4f184d" containerID="42b2247ba25a38fbcb1608511680e23b473144458869b6b0ca9406afd341f8aa" exitCode=0 Sep 18 21:14:22.968151 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:22.967768 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" event={"ID":"c943b701-413f-44d5-808c-61b9aa4f184d","Type":"ContainerDied","Data":"42b2247ba25a38fbcb1608511680e23b473144458869b6b0ca9406afd341f8aa"} Sep 18 21:14:22.969725 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:22.969701 2557 generic.go:356] "Generic (PLEG): container finished" podID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerID="6457601925f96cadf1b95d57778ce8fc3645e8b0419b2aa37a1170b87d9f2de2" exitCode=0 Sep 18 21:14:22.969816 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:22.969780 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" event={"ID":"8ff77b09-4860-469a-b6a0-5cc02d43ab67","Type":"ContainerDied","Data":"6457601925f96cadf1b95d57778ce8fc3645e8b0419b2aa37a1170b87d9f2de2"} Sep 18 21:14:22.971606 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:22.971586 2557 generic.go:356] "Generic (PLEG): container finished" podID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerID="96e8160f63690270272e2be1b6c69a107e46b5f559c2b30fd6a10c68be813f5d" exitCode=0 Sep 18 21:14:22.971716 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:22.971657 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" event={"ID":"ad86d541-9ac9-4963-92ae-45d57eb63675","Type":"ContainerDied","Data":"96e8160f63690270272e2be1b6c69a107e46b5f559c2b30fd6a10c68be813f5d"} Sep 18 21:14:22.973935 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:22.973911 2557 generic.go:356] "Generic (PLEG): container finished" podID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerID="96b5256f2f8cca6ec32ac4525c754020883181a0d37ed9dbdf7ff501f9de546b" exitCode=0 Sep 18 21:14:22.974032 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:22.973943 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" event={"ID":"dc8c6381-dd07-4db2-903b-2f848431ec95","Type":"ContainerDied","Data":"96b5256f2f8cca6ec32ac4525c754020883181a0d37ed9dbdf7ff501f9de546b"} Sep 18 21:14:23.982999 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:23.982967 2557 generic.go:356] "Generic (PLEG): container finished" podID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerID="941d0183ef51bc82e889094cf6936e457e5b134138e308975e3ba7a3c3e12ef3" exitCode=0 Sep 18 21:14:23.983440 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:23.983053 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" event={"ID":"ad86d541-9ac9-4963-92ae-45d57eb63675","Type":"ContainerDied","Data":"941d0183ef51bc82e889094cf6936e457e5b134138e308975e3ba7a3c3e12ef3"} Sep 18 21:14:24.124879 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.124851 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:24.153926 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.153897 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:24.156985 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.156962 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:24.213356 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.213300 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-util\") pod \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " Sep 18 21:14:24.213511 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.213417 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-bundle\") pod \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " Sep 18 21:14:24.213511 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.213481 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spnlw\" (UniqueName: \"kubernetes.io/projected/8ff77b09-4860-469a-b6a0-5cc02d43ab67-kube-api-access-spnlw\") pod \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\" (UID: \"8ff77b09-4860-469a-b6a0-5cc02d43ab67\") " Sep 18 21:14:24.214074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.214041 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-bundle" (OuterVolumeSpecName: "bundle") pod "8ff77b09-4860-469a-b6a0-5cc02d43ab67" (UID: "8ff77b09-4860-469a-b6a0-5cc02d43ab67"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:14:24.215586 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.215549 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff77b09-4860-469a-b6a0-5cc02d43ab67-kube-api-access-spnlw" (OuterVolumeSpecName: "kube-api-access-spnlw") pod "8ff77b09-4860-469a-b6a0-5cc02d43ab67" (UID: "8ff77b09-4860-469a-b6a0-5cc02d43ab67"). InnerVolumeSpecName "kube-api-access-spnlw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:14:24.218443 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.218423 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-util" (OuterVolumeSpecName: "util") pod "8ff77b09-4860-469a-b6a0-5cc02d43ab67" (UID: "8ff77b09-4860-469a-b6a0-5cc02d43ab67"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:14:24.315065 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.314973 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpxkg\" (UniqueName: \"kubernetes.io/projected/dc8c6381-dd07-4db2-903b-2f848431ec95-kube-api-access-gpxkg\") pod \"dc8c6381-dd07-4db2-903b-2f848431ec95\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " Sep 18 21:14:24.315065 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315032 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-bundle\") pod \"dc8c6381-dd07-4db2-903b-2f848431ec95\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " Sep 18 21:14:24.315266 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315070 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qx2l\" (UniqueName: \"kubernetes.io/projected/c943b701-413f-44d5-808c-61b9aa4f184d-kube-api-access-9qx2l\") pod \"c943b701-413f-44d5-808c-61b9aa4f184d\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " Sep 18 21:14:24.315266 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315168 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-util\") pod \"dc8c6381-dd07-4db2-903b-2f848431ec95\" (UID: \"dc8c6381-dd07-4db2-903b-2f848431ec95\") " Sep 18 21:14:24.315266 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315258 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-bundle\") pod \"c943b701-413f-44d5-808c-61b9aa4f184d\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " Sep 18 21:14:24.315451 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315282 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-util\") pod \"c943b701-413f-44d5-808c-61b9aa4f184d\" (UID: \"c943b701-413f-44d5-808c-61b9aa4f184d\") " Sep 18 21:14:24.315540 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315522 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-bundle\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:24.315594 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315545 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spnlw\" (UniqueName: \"kubernetes.io/projected/8ff77b09-4860-469a-b6a0-5cc02d43ab67-kube-api-access-spnlw\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:24.315594 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315558 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ff77b09-4860-469a-b6a0-5cc02d43ab67-util\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:24.315694 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315678 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-bundle" (OuterVolumeSpecName: "bundle") pod "dc8c6381-dd07-4db2-903b-2f848431ec95" (UID: "dc8c6381-dd07-4db2-903b-2f848431ec95"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:14:24.315855 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.315830 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-bundle" (OuterVolumeSpecName: "bundle") pod "c943b701-413f-44d5-808c-61b9aa4f184d" (UID: "c943b701-413f-44d5-808c-61b9aa4f184d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:14:24.317758 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.317738 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc8c6381-dd07-4db2-903b-2f848431ec95-kube-api-access-gpxkg" (OuterVolumeSpecName: "kube-api-access-gpxkg") pod "dc8c6381-dd07-4db2-903b-2f848431ec95" (UID: "dc8c6381-dd07-4db2-903b-2f848431ec95"). InnerVolumeSpecName "kube-api-access-gpxkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:14:24.317861 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.317765 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c943b701-413f-44d5-808c-61b9aa4f184d-kube-api-access-9qx2l" (OuterVolumeSpecName: "kube-api-access-9qx2l") pod "c943b701-413f-44d5-808c-61b9aa4f184d" (UID: "c943b701-413f-44d5-808c-61b9aa4f184d"). InnerVolumeSpecName "kube-api-access-9qx2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:14:24.320471 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.320442 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-util" (OuterVolumeSpecName: "util") pod "dc8c6381-dd07-4db2-903b-2f848431ec95" (UID: "dc8c6381-dd07-4db2-903b-2f848431ec95"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:14:24.321138 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.321118 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-util" (OuterVolumeSpecName: "util") pod "c943b701-413f-44d5-808c-61b9aa4f184d" (UID: "c943b701-413f-44d5-808c-61b9aa4f184d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:14:24.416342 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.416266 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-bundle\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:24.416342 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.416326 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c943b701-413f-44d5-808c-61b9aa4f184d-util\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:24.416342 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.416338 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpxkg\" (UniqueName: \"kubernetes.io/projected/dc8c6381-dd07-4db2-903b-2f848431ec95-kube-api-access-gpxkg\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:24.416342 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.416349 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-bundle\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:24.416342 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.416358 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qx2l\" (UniqueName: \"kubernetes.io/projected/c943b701-413f-44d5-808c-61b9aa4f184d-kube-api-access-9qx2l\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:24.416631 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.416365 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dc8c6381-dd07-4db2-903b-2f848431ec95-util\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:24.994563 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.994532 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" Sep 18 21:14:24.994563 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.994542 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4x9dw" event={"ID":"8ff77b09-4860-469a-b6a0-5cc02d43ab67","Type":"ContainerDied","Data":"6523aafb9971fddf3ad14c71457f2e82d0c699138cd1b5c705d4b9b584c574b4"} Sep 18 21:14:24.995044 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.994609 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6523aafb9971fddf3ad14c71457f2e82d0c699138cd1b5c705d4b9b584c574b4" Sep 18 21:14:24.996515 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.996483 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" event={"ID":"dc8c6381-dd07-4db2-903b-2f848431ec95","Type":"ContainerDied","Data":"51d6d4ca69eb6bc0e9eb5c7c30f0304dd8bb470812125d95cb1914d8efc0383c"} Sep 18 21:14:24.996515 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.996511 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6nln44" Sep 18 21:14:24.996690 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.996526 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51d6d4ca69eb6bc0e9eb5c7c30f0304dd8bb470812125d95cb1914d8efc0383c" Sep 18 21:14:24.998406 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.998385 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" Sep 18 21:14:24.998406 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.998397 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d77hn47" event={"ID":"c943b701-413f-44d5-808c-61b9aa4f184d","Type":"ContainerDied","Data":"903289666b8f292aa14188aceafc8f84860b30c5a4ebb8aee01b198dad013cd4"} Sep 18 21:14:24.998559 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:24.998435 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="903289666b8f292aa14188aceafc8f84860b30c5a4ebb8aee01b198dad013cd4" Sep 18 21:14:25.129277 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.129254 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:25.222857 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.222818 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-util\") pod \"ad86d541-9ac9-4963-92ae-45d57eb63675\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " Sep 18 21:14:25.223039 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.222883 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4vsv\" (UniqueName: \"kubernetes.io/projected/ad86d541-9ac9-4963-92ae-45d57eb63675-kube-api-access-g4vsv\") pod \"ad86d541-9ac9-4963-92ae-45d57eb63675\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " Sep 18 21:14:25.223039 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.222940 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-bundle\") pod \"ad86d541-9ac9-4963-92ae-45d57eb63675\" (UID: \"ad86d541-9ac9-4963-92ae-45d57eb63675\") " Sep 18 21:14:25.223621 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.223591 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-bundle" (OuterVolumeSpecName: "bundle") pod "ad86d541-9ac9-4963-92ae-45d57eb63675" (UID: "ad86d541-9ac9-4963-92ae-45d57eb63675"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:14:25.225183 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.225158 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad86d541-9ac9-4963-92ae-45d57eb63675-kube-api-access-g4vsv" (OuterVolumeSpecName: "kube-api-access-g4vsv") pod "ad86d541-9ac9-4963-92ae-45d57eb63675" (UID: "ad86d541-9ac9-4963-92ae-45d57eb63675"). InnerVolumeSpecName "kube-api-access-g4vsv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:14:25.229555 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.229465 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-util" (OuterVolumeSpecName: "util") pod "ad86d541-9ac9-4963-92ae-45d57eb63675" (UID: "ad86d541-9ac9-4963-92ae-45d57eb63675"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:14:25.323729 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.323623 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-bundle\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:25.323729 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.323665 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad86d541-9ac9-4963-92ae-45d57eb63675-util\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:25.323729 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:25.323677 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4vsv\" (UniqueName: \"kubernetes.io/projected/ad86d541-9ac9-4963-92ae-45d57eb63675-kube-api-access-g4vsv\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:14:26.005659 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:26.005630 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" Sep 18 21:14:26.006119 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:26.005628 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5dnwc" event={"ID":"ad86d541-9ac9-4963-92ae-45d57eb63675","Type":"ContainerDied","Data":"40ecdaf4dc4e335a11dab8a2b042a0449bdbd90fc440814e0391e6a4dbcfeef0"} Sep 18 21:14:26.006119 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:26.005752 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40ecdaf4dc4e335a11dab8a2b042a0449bdbd90fc440814e0391e6a4dbcfeef0" Sep 18 21:14:37.392745 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.392709 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-6f9dn"] Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393002 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerName="pull" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393012 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerName="pull" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393021 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c943b701-413f-44d5-808c-61b9aa4f184d" containerName="extract" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393027 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c943b701-413f-44d5-808c-61b9aa4f184d" containerName="extract" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393034 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerName="extract" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393040 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerName="extract" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393049 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerName="pull" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393054 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerName="pull" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393061 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerName="pull" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393066 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerName="pull" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393073 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c943b701-413f-44d5-808c-61b9aa4f184d" containerName="util" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393077 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c943b701-413f-44d5-808c-61b9aa4f184d" containerName="util" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393083 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerName="extract" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393088 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerName="extract" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393097 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerName="util" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393102 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerName="util" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393109 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerName="extract" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393114 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerName="extract" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393123 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerName="util" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393127 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerName="util" Sep 18 21:14:37.393121 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393133 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerName="util" Sep 18 21:14:37.393752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393139 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerName="util" Sep 18 21:14:37.393752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393145 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c943b701-413f-44d5-808c-61b9aa4f184d" containerName="pull" Sep 18 21:14:37.393752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393149 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c943b701-413f-44d5-808c-61b9aa4f184d" containerName="pull" Sep 18 21:14:37.393752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393192 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c943b701-413f-44d5-808c-61b9aa4f184d" containerName="extract" Sep 18 21:14:37.393752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393198 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff77b09-4860-469a-b6a0-5cc02d43ab67" containerName="extract" Sep 18 21:14:37.393752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393207 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad86d541-9ac9-4963-92ae-45d57eb63675" containerName="extract" Sep 18 21:14:37.393752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.393212 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc8c6381-dd07-4db2-903b-2f848431ec95" containerName="extract" Sep 18 21:14:37.396488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.396472 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" Sep 18 21:14:37.413167 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:14:37.413134 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Sep 18 21:14:37.413423 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:14:37.413398 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"authorino-operator-76b478bd78-6f9dn\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="d4886633-1e3c-48b0-8beb-6d8cd01d413d" pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" Sep 18 21:14:37.413638 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:14:37.413619 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Sep 18 21:14:37.415379 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:14:37.415352 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"authorino-operator-dockercfg-lnsx5\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lnsx5\"" type="*v1.Secret" Sep 18 21:14:37.427974 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.427947 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-6f9dn"] Sep 18 21:14:37.520043 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.520008 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhgll\" (UniqueName: \"kubernetes.io/projected/d4886633-1e3c-48b0-8beb-6d8cd01d413d-kube-api-access-xhgll\") pod \"authorino-operator-76b478bd78-6f9dn\" (UID: \"d4886633-1e3c-48b0-8beb-6d8cd01d413d\") " pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" Sep 18 21:14:37.621094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:37.621057 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhgll\" (UniqueName: \"kubernetes.io/projected/d4886633-1e3c-48b0-8beb-6d8cd01d413d-kube-api-access-xhgll\") pod \"authorino-operator-76b478bd78-6f9dn\" (UID: \"d4886633-1e3c-48b0-8beb-6d8cd01d413d\") " pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" Sep 18 21:14:38.633769 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:14:38.633722 2557 projected.go:291] Couldn't get configMap kuadrant-system/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Sep 18 21:14:38.672234 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:38.672195 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 18 21:14:38.674275 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:14:38.674257 2557 projected.go:196] Error preparing data for projected volume kube-api-access-xhgll for pod kuadrant-system/authorino-operator-76b478bd78-6f9dn: failed to sync configmap cache: timed out waiting for the condition Sep 18 21:14:38.674394 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:14:38.674375 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4886633-1e3c-48b0-8beb-6d8cd01d413d-kube-api-access-xhgll podName:d4886633-1e3c-48b0-8beb-6d8cd01d413d nodeName:}" failed. No retries permitted until 2026-09-18 21:14:39.174350791 +0000 UTC m=+521.822948373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xhgll" (UniqueName: "kubernetes.io/projected/d4886633-1e3c-48b0-8beb-6d8cd01d413d-kube-api-access-xhgll") pod "authorino-operator-76b478bd78-6f9dn" (UID: "d4886633-1e3c-48b0-8beb-6d8cd01d413d") : failed to sync configmap cache: timed out waiting for the condition Sep 18 21:14:38.830271 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:38.830238 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 18 21:14:38.982281 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:38.982193 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lnsx5\"" Sep 18 21:14:39.235352 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:39.235215 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhgll\" (UniqueName: \"kubernetes.io/projected/d4886633-1e3c-48b0-8beb-6d8cd01d413d-kube-api-access-xhgll\") pod \"authorino-operator-76b478bd78-6f9dn\" (UID: \"d4886633-1e3c-48b0-8beb-6d8cd01d413d\") " pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" Sep 18 21:14:39.237743 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:39.237716 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhgll\" (UniqueName: \"kubernetes.io/projected/d4886633-1e3c-48b0-8beb-6d8cd01d413d-kube-api-access-xhgll\") pod \"authorino-operator-76b478bd78-6f9dn\" (UID: \"d4886633-1e3c-48b0-8beb-6d8cd01d413d\") " pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" Sep 18 21:14:39.505632 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:39.505547 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" Sep 18 21:14:39.792980 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:39.792955 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-6f9dn"] Sep 18 21:14:39.795174 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:14:39.795140 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4886633_1e3c_48b0_8beb_6d8cd01d413d.slice/crio-4ac18488f299b499b36f892f1a5801c33333bc166c1796b8df7e6760d645b250 WatchSource:0}: Error finding container 4ac18488f299b499b36f892f1a5801c33333bc166c1796b8df7e6760d645b250: Status 404 returned error can't find the container with id 4ac18488f299b499b36f892f1a5801c33333bc166c1796b8df7e6760d645b250 Sep 18 21:14:40.071432 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:40.071343 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" event={"ID":"d4886633-1e3c-48b0-8beb-6d8cd01d413d","Type":"ContainerStarted","Data":"4ac18488f299b499b36f892f1a5801c33333bc166c1796b8df7e6760d645b250"} Sep 18 21:14:41.701059 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.701021 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t"] Sep 18 21:14:41.707058 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.707036 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:41.709835 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.709808 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4vrw6\"" Sep 18 21:14:41.730226 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.730194 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t"] Sep 18 21:14:41.760570 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.760538 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdkp4\" (UniqueName: \"kubernetes.io/projected/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-kube-api-access-fdkp4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" (UID: \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:41.760716 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.760590 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" (UID: \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:41.861240 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.861208 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" (UID: \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:41.861467 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.861330 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdkp4\" (UniqueName: \"kubernetes.io/projected/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-kube-api-access-fdkp4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" (UID: \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:41.861661 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.861631 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" (UID: \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:41.874040 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:41.874013 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdkp4\" (UniqueName: \"kubernetes.io/projected/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-kube-api-access-fdkp4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" (UID: \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:42.018129 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:42.018045 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:42.326089 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:42.325915 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t"] Sep 18 21:14:42.328920 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:14:42.328889 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d6dcd4_8da1_4aaf_bc0f_d19e64c0135e.slice/crio-918c879fbda170e66c9d308d02a7763a6a67e96b556d02eed857598f91b723ab WatchSource:0}: Error finding container 918c879fbda170e66c9d308d02a7763a6a67e96b556d02eed857598f91b723ab: Status 404 returned error can't find the container with id 918c879fbda170e66c9d308d02a7763a6a67e96b556d02eed857598f91b723ab Sep 18 21:14:43.091145 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.091095 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" event={"ID":"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e","Type":"ContainerStarted","Data":"918c879fbda170e66c9d308d02a7763a6a67e96b556d02eed857598f91b723ab"} Sep 18 21:14:43.093329 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.093279 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" event={"ID":"d4886633-1e3c-48b0-8beb-6d8cd01d413d","Type":"ContainerStarted","Data":"f6164b41548ada441d4b5a1963cdf1d0b62c1817d7b08376ed79d5be887ce443"} Sep 18 21:14:43.093478 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.093401 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" Sep 18 21:14:43.129829 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.129761 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" podStartSLOduration=3.878774075 podStartE2EDuration="6.129738403s" totalImagesPullingTime="2.250964328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:14:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:14:43.128838945 +0000 UTC m=+525.777436536" watchObservedRunningTime="2026-09-18 21:14:43.129738403 +0000 UTC m=+525.778335995" Sep 18 21:14:43.176679 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.176648 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc"] Sep 18 21:14:43.179964 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.179940 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" Sep 18 21:14:43.183921 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.183898 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 18 21:14:43.184494 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.184473 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-mdv65\"" Sep 18 21:14:43.206022 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.205995 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc"] Sep 18 21:14:43.272126 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.272084 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlr28\" (UniqueName: \"kubernetes.io/projected/23c92796-0e77-4002-ab26-cf79366e832f-kube-api-access-zlr28\") pod \"dns-operator-controller-manager-7b84cd7c9c-gzqzc\" (UID: \"23c92796-0e77-4002-ab26-cf79366e832f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" Sep 18 21:14:43.373008 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.372963 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlr28\" (UniqueName: \"kubernetes.io/projected/23c92796-0e77-4002-ab26-cf79366e832f-kube-api-access-zlr28\") pod \"dns-operator-controller-manager-7b84cd7c9c-gzqzc\" (UID: \"23c92796-0e77-4002-ab26-cf79366e832f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" Sep 18 21:14:43.391044 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.391009 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlr28\" (UniqueName: \"kubernetes.io/projected/23c92796-0e77-4002-ab26-cf79366e832f-kube-api-access-zlr28\") pod \"dns-operator-controller-manager-7b84cd7c9c-gzqzc\" (UID: \"23c92796-0e77-4002-ab26-cf79366e832f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" Sep 18 21:14:43.493011 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.492972 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" Sep 18 21:14:43.783583 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:43.783540 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc"] Sep 18 21:14:43.784931 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:14:43.784896 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c92796_0e77_4002_ab26_cf79366e832f.slice/crio-6702673fdb8e65df7ae7635a169bdcbff8d2d2d3611eef79e01878d7352c016a WatchSource:0}: Error finding container 6702673fdb8e65df7ae7635a169bdcbff8d2d2d3611eef79e01878d7352c016a: Status 404 returned error can't find the container with id 6702673fdb8e65df7ae7635a169bdcbff8d2d2d3611eef79e01878d7352c016a Sep 18 21:14:44.101432 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:44.101407 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" event={"ID":"23c92796-0e77-4002-ab26-cf79366e832f","Type":"ContainerStarted","Data":"6702673fdb8e65df7ae7635a169bdcbff8d2d2d3611eef79e01878d7352c016a"} Sep 18 21:14:45.761753 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:45.761718 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl"] Sep 18 21:14:45.765270 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:45.765245 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:14:45.768548 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:45.768521 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-65kxx\"" Sep 18 21:14:45.781211 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:45.781183 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl"] Sep 18 21:14:45.792489 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:45.792447 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfk6k\" (UniqueName: \"kubernetes.io/projected/7ebbab64-1e00-4d74-9f07-b391b31cde5d-kube-api-access-kfk6k\") pod \"limitador-operator-controller-manager-74495c69f8-7mkjl\" (UID: \"7ebbab64-1e00-4d74-9f07-b391b31cde5d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:14:45.892969 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:45.892938 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfk6k\" (UniqueName: \"kubernetes.io/projected/7ebbab64-1e00-4d74-9f07-b391b31cde5d-kube-api-access-kfk6k\") pod \"limitador-operator-controller-manager-74495c69f8-7mkjl\" (UID: \"7ebbab64-1e00-4d74-9f07-b391b31cde5d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:14:45.913073 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:45.913031 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfk6k\" (UniqueName: \"kubernetes.io/projected/7ebbab64-1e00-4d74-9f07-b391b31cde5d-kube-api-access-kfk6k\") pod \"limitador-operator-controller-manager-74495c69f8-7mkjl\" (UID: \"7ebbab64-1e00-4d74-9f07-b391b31cde5d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:14:46.078186 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:46.078106 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:14:46.571579 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:46.571554 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl"] Sep 18 21:14:46.573190 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:14:46.573162 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebbab64_1e00_4d74_9f07_b391b31cde5d.slice/crio-4025fc5c940c016c34c62f7ee003fff33a804205e7488303c57864f5c3f9d698 WatchSource:0}: Error finding container 4025fc5c940c016c34c62f7ee003fff33a804205e7488303c57864f5c3f9d698: Status 404 returned error can't find the container with id 4025fc5c940c016c34c62f7ee003fff33a804205e7488303c57864f5c3f9d698 Sep 18 21:14:47.120392 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:47.120349 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" event={"ID":"23c92796-0e77-4002-ab26-cf79366e832f","Type":"ContainerStarted","Data":"6affb3cff64aae6b87500ec2352d7ac790ce925313fb9119eda58e3dda16178d"} Sep 18 21:14:47.120843 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:47.120530 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" Sep 18 21:14:47.121845 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:47.121823 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" event={"ID":"7ebbab64-1e00-4d74-9f07-b391b31cde5d","Type":"ContainerStarted","Data":"4025fc5c940c016c34c62f7ee003fff33a804205e7488303c57864f5c3f9d698"} Sep 18 21:14:47.123700 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:47.123677 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" event={"ID":"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e","Type":"ContainerStarted","Data":"2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1"} Sep 18 21:14:47.123827 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:47.123808 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:47.165715 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:47.165657 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" podStartSLOduration=1.6585143919999998 podStartE2EDuration="4.165640133s" totalImagesPullingTime="2.507125741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:14:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:14:47.162929905 +0000 UTC m=+529.811527497" watchObservedRunningTime="2026-09-18 21:14:47.165640133 +0000 UTC m=+529.814237724" Sep 18 21:14:47.218363 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:47.218283 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" podStartSLOduration=2.252380935 podStartE2EDuration="6.218258993s" totalImagesPullingTime="3.965878058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:14:47.215910049 +0000 UTC m=+529.864507642" watchObservedRunningTime="2026-09-18 21:14:47.218258993 +0000 UTC m=+529.866856585" Sep 18 21:14:48.133977 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:48.133940 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" event={"ID":"7ebbab64-1e00-4d74-9f07-b391b31cde5d","Type":"ContainerStarted","Data":"ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82"} Sep 18 21:14:48.134446 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:48.133992 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:14:48.178096 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:48.178036 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" podStartSLOduration=1.779681025 podStartE2EDuration="3.178018983s" totalImagesPullingTime="1.398337958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:14:48.176403743 +0000 UTC m=+530.825001333" watchObservedRunningTime="2026-09-18 21:14:48.178018983 +0000 UTC m=+530.826616574" Sep 18 21:14:49.353422 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.353385 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl"] Sep 18 21:14:49.357097 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.357077 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:49.365129 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.365107 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 18 21:14:49.365129 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.365121 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gt9tr\"" Sep 18 21:14:49.365343 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.365125 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 18 21:14:49.372382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.372359 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl"] Sep 18 21:14:49.419618 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.419585 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkk8t\" (UniqueName: \"kubernetes.io/projected/9442400d-1fde-4963-bddf-d1bc61c9e466-kube-api-access-zkk8t\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:49.419618 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.419621 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9442400d-1fde-4963-bddf-d1bc61c9e466-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:49.419820 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.419696 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9442400d-1fde-4963-bddf-d1bc61c9e466-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:49.521007 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.520969 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkk8t\" (UniqueName: \"kubernetes.io/projected/9442400d-1fde-4963-bddf-d1bc61c9e466-kube-api-access-zkk8t\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:49.521007 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.521008 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9442400d-1fde-4963-bddf-d1bc61c9e466-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:49.521234 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.521046 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9442400d-1fde-4963-bddf-d1bc61c9e466-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:49.521234 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:14:49.521174 2557 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Sep 18 21:14:49.521234 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:14:49.521234 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9442400d-1fde-4963-bddf-d1bc61c9e466-plugin-serving-cert podName:9442400d-1fde-4963-bddf-d1bc61c9e466 nodeName:}" failed. No retries permitted until 2026-09-18 21:14:50.021214866 +0000 UTC m=+532.669812435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/9442400d-1fde-4963-bddf-d1bc61c9e466-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-ql4zl" (UID: "9442400d-1fde-4963-bddf-d1bc61c9e466") : secret "plugin-serving-cert" not found Sep 18 21:14:49.521740 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.521721 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9442400d-1fde-4963-bddf-d1bc61c9e466-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:49.542165 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:49.542133 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkk8t\" (UniqueName: \"kubernetes.io/projected/9442400d-1fde-4963-bddf-d1bc61c9e466-kube-api-access-zkk8t\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:50.024916 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:50.024880 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9442400d-1fde-4963-bddf-d1bc61c9e466-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:50.027400 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:50.027375 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9442400d-1fde-4963-bddf-d1bc61c9e466-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-ql4zl\" (UID: \"9442400d-1fde-4963-bddf-d1bc61c9e466\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:50.265928 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:50.265841 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" Sep 18 21:14:50.542994 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:50.542968 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl"] Sep 18 21:14:50.545082 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:14:50.545053 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9442400d_1fde_4963_bddf_d1bc61c9e466.slice/crio-a1eea1f6365fca434002c78cd49e86ee5e7cdaf42d8e267cc57362e183c3aec4 WatchSource:0}: Error finding container a1eea1f6365fca434002c78cd49e86ee5e7cdaf42d8e267cc57362e183c3aec4: Status 404 returned error can't find the container with id a1eea1f6365fca434002c78cd49e86ee5e7cdaf42d8e267cc57362e183c3aec4 Sep 18 21:14:51.151384 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:51.151345 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" event={"ID":"9442400d-1fde-4963-bddf-d1bc61c9e466","Type":"ContainerStarted","Data":"a1eea1f6365fca434002c78cd49e86ee5e7cdaf42d8e267cc57362e183c3aec4"} Sep 18 21:14:54.104131 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:54.104100 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-6f9dn" Sep 18 21:14:55.179202 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:55.179162 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" event={"ID":"9442400d-1fde-4963-bddf-d1bc61c9e466","Type":"ContainerStarted","Data":"edcb80771cbc1e814ce3b4070435d66f58295520065957dd1d0c2b88fc1bb819"} Sep 18 21:14:55.217812 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:55.217752 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-ql4zl" podStartSLOduration=1.781393242 podStartE2EDuration="6.217736437s" totalImagesPullingTime="4.436343195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:14:55.216459471 +0000 UTC m=+537.865057063" watchObservedRunningTime="2026-09-18 21:14:55.217736437 +0000 UTC m=+537.866334062" Sep 18 21:14:58.136071 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:58.136037 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:14:58.136488 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:58.136218 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-gzqzc" Sep 18 21:14:59.141469 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:14:59.141439 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:15:01.206875 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.206835 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t"] Sep 18 21:15:01.207457 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.207064 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" containerName="manager" containerID="cri-o://2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1" gracePeriod=2 Sep 18 21:15:01.211653 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.211627 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t"] Sep 18 21:15:01.222607 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.222560 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:15:01.236571 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.236531 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl"] Sep 18 21:15:01.236934 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.236900 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" podUID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" containerName="manager" containerID="cri-o://ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82" gracePeriod=2 Sep 18 21:15:01.244061 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.244030 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:15:01.249677 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.249648 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl"] Sep 18 21:15:01.278554 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.278525 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf"] Sep 18 21:15:01.278823 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.278811 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" containerName="manager" Sep 18 21:15:01.278823 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.278824 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" containerName="manager" Sep 18 21:15:01.278932 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.278837 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" containerName="manager" Sep 18 21:15:01.278932 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.278843 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" containerName="manager" Sep 18 21:15:01.278932 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.278902 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" containerName="manager" Sep 18 21:15:01.278932 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.278910 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" containerName="manager" Sep 18 21:15:01.281717 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.281700 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:01.295531 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.295501 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr"] Sep 18 21:15:01.299291 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.299271 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" Sep 18 21:15:01.314425 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.314367 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf"] Sep 18 21:15:01.320306 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.320280 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff4040af-bb72-4d7e-8aa9-78ac51de2903-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-scglf\" (UID: \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:01.320437 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.320417 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t42nl\" (UniqueName: \"kubernetes.io/projected/ff4040af-bb72-4d7e-8aa9-78ac51de2903-kube-api-access-t42nl\") pod \"kuadrant-operator-controller-manager-76f8f686bf-scglf\" (UID: \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:01.332602 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.332570 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr"] Sep 18 21:15:01.352195 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.352127 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7mkjl\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:15:01.355853 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.355756 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:15:01.360698 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.360658 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7mkjl\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:15:01.412090 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.411984 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:15:01.421130 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.421102 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff4040af-bb72-4d7e-8aa9-78ac51de2903-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-scglf\" (UID: \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:01.421294 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.421174 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44gwl\" (UniqueName: \"kubernetes.io/projected/42c966c3-eb8f-4dc9-b481-35e977252077-kube-api-access-44gwl\") pod \"limitador-operator-controller-manager-74495c69f8-td8fr\" (UID: \"42c966c3-eb8f-4dc9-b481-35e977252077\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" Sep 18 21:15:01.421294 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.421196 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t42nl\" (UniqueName: \"kubernetes.io/projected/ff4040af-bb72-4d7e-8aa9-78ac51de2903-kube-api-access-t42nl\") pod \"kuadrant-operator-controller-manager-76f8f686bf-scglf\" (UID: \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:01.421578 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.421554 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff4040af-bb72-4d7e-8aa9-78ac51de2903-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-scglf\" (UID: \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:01.451252 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.451207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t42nl\" (UniqueName: \"kubernetes.io/projected/ff4040af-bb72-4d7e-8aa9-78ac51de2903-kube-api-access-t42nl\") pod \"kuadrant-operator-controller-manager-76f8f686bf-scglf\" (UID: \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:01.477934 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.477912 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:15:01.480401 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.480381 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:15:01.480597 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.480569 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:15:01.482892 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.482872 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7mkjl\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:15:01.486181 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.486152 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-ksm5t\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:15:01.491041 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:01.491019 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7mkjl\" is forbidden: User \"system:node:ip-10-0-129-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-163.ec2.internal' and this object" podUID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:15:01.522501 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.522466 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfk6k\" (UniqueName: \"kubernetes.io/projected/7ebbab64-1e00-4d74-9f07-b391b31cde5d-kube-api-access-kfk6k\") pod \"7ebbab64-1e00-4d74-9f07-b391b31cde5d\" (UID: \"7ebbab64-1e00-4d74-9f07-b391b31cde5d\") " Sep 18 21:15:01.522647 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.522565 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-extensions-socket-volume\") pod \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\" (UID: \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\") " Sep 18 21:15:01.522647 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.522603 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdkp4\" (UniqueName: \"kubernetes.io/projected/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-kube-api-access-fdkp4\") pod \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\" (UID: \"47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e\") " Sep 18 21:15:01.522735 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.522664 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gwl\" (UniqueName: \"kubernetes.io/projected/42c966c3-eb8f-4dc9-b481-35e977252077-kube-api-access-44gwl\") pod \"limitador-operator-controller-manager-74495c69f8-td8fr\" (UID: \"42c966c3-eb8f-4dc9-b481-35e977252077\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" Sep 18 21:15:01.522948 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.522925 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" (UID: "47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:15:01.524744 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.524723 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ebbab64-1e00-4d74-9f07-b391b31cde5d-kube-api-access-kfk6k" (OuterVolumeSpecName: "kube-api-access-kfk6k") pod "7ebbab64-1e00-4d74-9f07-b391b31cde5d" (UID: "7ebbab64-1e00-4d74-9f07-b391b31cde5d"). InnerVolumeSpecName "kube-api-access-kfk6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:15:01.524818 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.524797 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-kube-api-access-fdkp4" (OuterVolumeSpecName: "kube-api-access-fdkp4") pod "47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" (UID: "47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e"). InnerVolumeSpecName "kube-api-access-fdkp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:15:01.536851 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.536821 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44gwl\" (UniqueName: \"kubernetes.io/projected/42c966c3-eb8f-4dc9-b481-35e977252077-kube-api-access-44gwl\") pod \"limitador-operator-controller-manager-74495c69f8-td8fr\" (UID: \"42c966c3-eb8f-4dc9-b481-35e977252077\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" Sep 18 21:15:01.623558 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.623519 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-extensions-socket-volume\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:15:01.623558 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.623550 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdkp4\" (UniqueName: \"kubernetes.io/projected/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e-kube-api-access-fdkp4\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:15:01.623558 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.623559 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfk6k\" (UniqueName: \"kubernetes.io/projected/7ebbab64-1e00-4d74-9f07-b391b31cde5d-kube-api-access-kfk6k\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:15:01.624384 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.624371 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:01.635752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.635731 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" Sep 18 21:15:01.860126 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.860087 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" path="/var/lib/kubelet/pods/47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e/volumes" Sep 18 21:15:01.860447 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.860434 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" path="/var/lib/kubelet/pods/7ebbab64-1e00-4d74-9f07-b391b31cde5d/volumes" Sep 18 21:15:01.912251 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.912172 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf"] Sep 18 21:15:01.914967 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:15:01.914942 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4040af_bb72_4d7e_8aa9_78ac51de2903.slice/crio-edde8ceabbcd955c22732b7214054fd2c20a95073325d7862074cfc3a1dd2d18 WatchSource:0}: Error finding container edde8ceabbcd955c22732b7214054fd2c20a95073325d7862074cfc3a1dd2d18: Status 404 returned error can't find the container with id edde8ceabbcd955c22732b7214054fd2c20a95073325d7862074cfc3a1dd2d18 Sep 18 21:15:01.953561 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:01.953530 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr"] Sep 18 21:15:01.956824 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:15:01.956794 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c966c3_eb8f_4dc9_b481_35e977252077.slice/crio-f22b93ba2af7eaa0849289a8dabce5285507fddd99afe900c2b205ad403fa01e WatchSource:0}: Error finding container f22b93ba2af7eaa0849289a8dabce5285507fddd99afe900c2b205ad403fa01e: Status 404 returned error can't find the container with id f22b93ba2af7eaa0849289a8dabce5285507fddd99afe900c2b205ad403fa01e Sep 18 21:15:02.219586 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.219495 2557 generic.go:356] "Generic (PLEG): container finished" podID="7ebbab64-1e00-4d74-9f07-b391b31cde5d" containerID="ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82" exitCode=0 Sep 18 21:15:02.219586 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.219544 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7mkjl" Sep 18 21:15:02.219586 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.219551 2557 scope.go:117] "RemoveContainer" containerID="ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82" Sep 18 21:15:02.221725 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.221698 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" event={"ID":"ff4040af-bb72-4d7e-8aa9-78ac51de2903","Type":"ContainerStarted","Data":"fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692"} Sep 18 21:15:02.221843 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.221752 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" event={"ID":"ff4040af-bb72-4d7e-8aa9-78ac51de2903","Type":"ContainerStarted","Data":"edde8ceabbcd955c22732b7214054fd2c20a95073325d7862074cfc3a1dd2d18"} Sep 18 21:15:02.221843 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.221813 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:02.223809 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.223781 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" event={"ID":"42c966c3-eb8f-4dc9-b481-35e977252077","Type":"ContainerStarted","Data":"59b1edc8f621944fddcbd2f50de0498a9d273f42fb0dc23a6a7605b427960a6b"} Sep 18 21:15:02.223903 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.223822 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" event={"ID":"42c966c3-eb8f-4dc9-b481-35e977252077","Type":"ContainerStarted","Data":"f22b93ba2af7eaa0849289a8dabce5285507fddd99afe900c2b205ad403fa01e"} Sep 18 21:15:02.223961 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.223916 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" Sep 18 21:15:02.225190 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.225166 2557 generic.go:356] "Generic (PLEG): container finished" podID="47d6dcd4-8da1-4aaf-bc0f-d19e64c0135e" containerID="2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1" exitCode=0 Sep 18 21:15:02.225190 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.225198 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-ksm5t" Sep 18 21:15:02.242479 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.242457 2557 scope.go:117] "RemoveContainer" containerID="ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82" Sep 18 21:15:02.242804 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:02.242785 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82\": container with ID starting with ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82 not found: ID does not exist" containerID="ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82" Sep 18 21:15:02.242869 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.242813 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82"} err="failed to get container status \"ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82\": rpc error: code = NotFound desc = could not find container \"ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82\": container with ID starting with ab5e0b9391b2218a94559042c619c140ccc3cff321bc33bf6d9d6481ce329a82 not found: ID does not exist" Sep 18 21:15:02.242869 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.242832 2557 scope.go:117] "RemoveContainer" containerID="2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1" Sep 18 21:15:02.251705 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.251683 2557 scope.go:117] "RemoveContainer" containerID="2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1" Sep 18 21:15:02.252010 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:02.251989 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1\": container with ID starting with 2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1 not found: ID does not exist" containerID="2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1" Sep 18 21:15:02.252065 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.252017 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1"} err="failed to get container status \"2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1\": rpc error: code = NotFound desc = could not find container \"2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1\": container with ID starting with 2a6b3574ac8f53e6d4b07b05dff7f8284cad28eb132fd8a19fb77ed439f955b1 not found: ID does not exist" Sep 18 21:15:02.256754 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.256700 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" podStartSLOduration=1.256681704 podStartE2EDuration="1.256681704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:15:02.254476513 +0000 UTC m=+544.903074103" watchObservedRunningTime="2026-09-18 21:15:02.256681704 +0000 UTC m=+544.905279296" Sep 18 21:15:02.280831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.280778 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" podStartSLOduration=1.280762392 podStartE2EDuration="1.280762392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:15:02.28034444 +0000 UTC m=+544.928942031" watchObservedRunningTime="2026-09-18 21:15:02.280762392 +0000 UTC m=+544.929359983" Sep 18 21:15:02.770637 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.770599 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj"] Sep 18 21:15:02.775374 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.775354 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:02.834643 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.834609 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdqrb\" (UniqueName: \"kubernetes.io/projected/aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa-kube-api-access-kdqrb\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7hzcj\" (UID: \"aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:02.834832 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.834659 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7hzcj\" (UID: \"aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:02.880044 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.880011 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj"] Sep 18 21:15:02.935566 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.935528 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdqrb\" (UniqueName: \"kubernetes.io/projected/aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa-kube-api-access-kdqrb\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7hzcj\" (UID: \"aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:02.935752 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.935591 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7hzcj\" (UID: \"aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:02.935990 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.935968 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7hzcj\" (UID: \"aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:02.950145 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:02.950109 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdqrb\" (UniqueName: \"kubernetes.io/projected/aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa-kube-api-access-kdqrb\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7hzcj\" (UID: \"aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:03.084673 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:03.084583 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:03.365563 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:03.365538 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj"] Sep 18 21:15:03.367242 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:15:03.367214 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa15d486_0ac7_4b4a_b9d4_c7796d2ae2fa.slice/crio-ca3a50a5d050b55f870a5aff72f56692165225b547a6bdf5c809b36aae9326d7 WatchSource:0}: Error finding container ca3a50a5d050b55f870a5aff72f56692165225b547a6bdf5c809b36aae9326d7: Status 404 returned error can't find the container with id ca3a50a5d050b55f870a5aff72f56692165225b547a6bdf5c809b36aae9326d7 Sep 18 21:15:04.240493 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:04.240461 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" event={"ID":"aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa","Type":"ContainerStarted","Data":"547c452bbbb7c105a02340495d6969a65dcd892e31a3d8c8f695a06f8d1cc764"} Sep 18 21:15:04.240675 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:04.240511 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" event={"ID":"aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa","Type":"ContainerStarted","Data":"ca3a50a5d050b55f870a5aff72f56692165225b547a6bdf5c809b36aae9326d7"} Sep 18 21:15:04.240675 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:04.240599 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:04.270870 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:04.270814 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" podStartSLOduration=2.270795835 podStartE2EDuration="2.270795835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:15:04.268702674 +0000 UTC m=+546.917300264" watchObservedRunningTime="2026-09-18 21:15:04.270795835 +0000 UTC m=+546.919393482" Sep 18 21:15:13.236361 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:13.236307 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:13.236748 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:13.236398 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-td8fr" Sep 18 21:15:15.248462 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.248430 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7hzcj" Sep 18 21:15:15.332395 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.332363 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf"] Sep 18 21:15:15.332602 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.332561 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" podUID="ff4040af-bb72-4d7e-8aa9-78ac51de2903" containerName="manager" containerID="cri-o://fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692" gracePeriod=10 Sep 18 21:15:15.628828 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.628798 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:15.752613 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.752584 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff4040af-bb72-4d7e-8aa9-78ac51de2903-extensions-socket-volume\") pod \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\" (UID: \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\") " Sep 18 21:15:15.752760 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.752664 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t42nl\" (UniqueName: \"kubernetes.io/projected/ff4040af-bb72-4d7e-8aa9-78ac51de2903-kube-api-access-t42nl\") pod \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\" (UID: \"ff4040af-bb72-4d7e-8aa9-78ac51de2903\") " Sep 18 21:15:15.753089 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.753056 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff4040af-bb72-4d7e-8aa9-78ac51de2903-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ff4040af-bb72-4d7e-8aa9-78ac51de2903" (UID: "ff4040af-bb72-4d7e-8aa9-78ac51de2903"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 21:15:15.754747 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.754725 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff4040af-bb72-4d7e-8aa9-78ac51de2903-kube-api-access-t42nl" (OuterVolumeSpecName: "kube-api-access-t42nl") pod "ff4040af-bb72-4d7e-8aa9-78ac51de2903" (UID: "ff4040af-bb72-4d7e-8aa9-78ac51de2903"). InnerVolumeSpecName "kube-api-access-t42nl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:15:15.853510 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.853488 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t42nl\" (UniqueName: \"kubernetes.io/projected/ff4040af-bb72-4d7e-8aa9-78ac51de2903-kube-api-access-t42nl\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:15:15.853510 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:15.853512 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff4040af-bb72-4d7e-8aa9-78ac51de2903-extensions-socket-volume\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:15:16.304203 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:16.304123 2557 generic.go:356] "Generic (PLEG): container finished" podID="ff4040af-bb72-4d7e-8aa9-78ac51de2903" containerID="fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692" exitCode=0 Sep 18 21:15:16.304203 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:16.304168 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" Sep 18 21:15:16.304651 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:16.304209 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" event={"ID":"ff4040af-bb72-4d7e-8aa9-78ac51de2903","Type":"ContainerDied","Data":"fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692"} Sep 18 21:15:16.304651 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:16.304269 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf" event={"ID":"ff4040af-bb72-4d7e-8aa9-78ac51de2903","Type":"ContainerDied","Data":"edde8ceabbcd955c22732b7214054fd2c20a95073325d7862074cfc3a1dd2d18"} Sep 18 21:15:16.304651 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:16.304297 2557 scope.go:117] "RemoveContainer" containerID="fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692" Sep 18 21:15:16.315412 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:16.315391 2557 scope.go:117] "RemoveContainer" containerID="fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692" Sep 18 21:15:16.315717 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:16.315692 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692\": container with ID starting with fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692 not found: ID does not exist" containerID="fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692" Sep 18 21:15:16.315808 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:16.315728 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692"} err="failed to get container status \"fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692\": rpc error: code = NotFound desc = could not find container \"fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692\": container with ID starting with fbbe460d0ef2c96c8534aad2f2401232bf88caee7f73b25071047d9900e59692 not found: ID does not exist" Sep 18 21:15:16.346136 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:16.346105 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf"] Sep 18 21:15:16.348074 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:16.348053 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-scglf"] Sep 18 21:15:17.861386 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:17.861352 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff4040af-bb72-4d7e-8aa9-78ac51de2903" path="/var/lib/kubelet/pods/ff4040af-bb72-4d7e-8aa9-78ac51de2903/volumes" Sep 18 21:15:33.965514 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:33.965479 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pwvr4"] Sep 18 21:15:33.965902 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:33.965789 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff4040af-bb72-4d7e-8aa9-78ac51de2903" containerName="manager" Sep 18 21:15:33.965902 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:33.965800 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff4040af-bb72-4d7e-8aa9-78ac51de2903" containerName="manager" Sep 18 21:15:33.965902 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:33.965859 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff4040af-bb72-4d7e-8aa9-78ac51de2903" containerName="manager" Sep 18 21:15:33.983334 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:33.983278 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:33.987788 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:33.987754 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pwvr4"] Sep 18 21:15:33.989104 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:33.989079 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 18 21:15:34.051408 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:34.051374 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pwvr4"] Sep 18 21:15:34.116713 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:34.116670 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/42429754-09cc-4bea-b0ce-c2318081462c-config-file\") pod \"limitador-limitador-8588749977-pwvr4\" (UID: \"42429754-09cc-4bea-b0ce-c2318081462c\") " pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:34.116888 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:34.116721 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkwn6\" (UniqueName: \"kubernetes.io/projected/42429754-09cc-4bea-b0ce-c2318081462c-kube-api-access-rkwn6\") pod \"limitador-limitador-8588749977-pwvr4\" (UID: \"42429754-09cc-4bea-b0ce-c2318081462c\") " pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:34.217558 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:34.217460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/42429754-09cc-4bea-b0ce-c2318081462c-config-file\") pod \"limitador-limitador-8588749977-pwvr4\" (UID: \"42429754-09cc-4bea-b0ce-c2318081462c\") " pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:34.217558 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:34.217509 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkwn6\" (UniqueName: \"kubernetes.io/projected/42429754-09cc-4bea-b0ce-c2318081462c-kube-api-access-rkwn6\") pod \"limitador-limitador-8588749977-pwvr4\" (UID: \"42429754-09cc-4bea-b0ce-c2318081462c\") " pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:34.218094 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:34.218070 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/42429754-09cc-4bea-b0ce-c2318081462c-config-file\") pod \"limitador-limitador-8588749977-pwvr4\" (UID: \"42429754-09cc-4bea-b0ce-c2318081462c\") " pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:34.228721 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:34.228688 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkwn6\" (UniqueName: \"kubernetes.io/projected/42429754-09cc-4bea-b0ce-c2318081462c-kube-api-access-rkwn6\") pod \"limitador-limitador-8588749977-pwvr4\" (UID: \"42429754-09cc-4bea-b0ce-c2318081462c\") " pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:34.295721 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:34.295686 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:34.626156 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:15:34.626126 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42429754_09cc_4bea_b0ce_c2318081462c.slice/crio-e7532d3d5a1e4bcfdaaa27b5452d3f2d63e3f82fa03d67bf746798697faab209 WatchSource:0}: Error finding container e7532d3d5a1e4bcfdaaa27b5452d3f2d63e3f82fa03d67bf746798697faab209: Status 404 returned error can't find the container with id e7532d3d5a1e4bcfdaaa27b5452d3f2d63e3f82fa03d67bf746798697faab209 Sep 18 21:15:34.627229 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:34.627198 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pwvr4"] Sep 18 21:15:35.408306 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:35.408262 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" event={"ID":"42429754-09cc-4bea-b0ce-c2318081462c","Type":"ContainerStarted","Data":"e7532d3d5a1e4bcfdaaa27b5452d3f2d63e3f82fa03d67bf746798697faab209"} Sep 18 21:15:36.414819 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:36.414780 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" event={"ID":"42429754-09cc-4bea-b0ce-c2318081462c","Type":"ContainerStarted","Data":"e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b"} Sep 18 21:15:36.415251 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:36.414934 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:36.451649 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:36.451591 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" podStartSLOduration=2.277476067 podStartE2EDuration="3.451574664s" totalImagesPullingTime="1.174098597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:15:36.449252031 +0000 UTC m=+579.097849621" watchObservedRunningTime="2026-09-18 21:15:36.451574664 +0000 UTC m=+579.100172257" Sep 18 21:15:47.421096 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:47.421062 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:48.904226 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:48.904172 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pwvr4"] Sep 18 21:15:48.904799 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:48.904435 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" podUID="42429754-09cc-4bea-b0ce-c2318081462c" containerName="limitador" containerID="cri-o://e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b" gracePeriod=30 Sep 18 21:15:49.458288 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.458260 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:49.482352 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.482246 2557 generic.go:356] "Generic (PLEG): container finished" podID="42429754-09cc-4bea-b0ce-c2318081462c" containerID="e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b" exitCode=0 Sep 18 21:15:49.482352 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.482294 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" Sep 18 21:15:49.482352 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.482345 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" event={"ID":"42429754-09cc-4bea-b0ce-c2318081462c","Type":"ContainerDied","Data":"e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b"} Sep 18 21:15:49.482560 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.482424 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pwvr4" event={"ID":"42429754-09cc-4bea-b0ce-c2318081462c","Type":"ContainerDied","Data":"e7532d3d5a1e4bcfdaaa27b5452d3f2d63e3f82fa03d67bf746798697faab209"} Sep 18 21:15:49.482560 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.482449 2557 scope.go:117] "RemoveContainer" containerID="e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b" Sep 18 21:15:49.492859 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.492837 2557 scope.go:117] "RemoveContainer" containerID="e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b" Sep 18 21:15:49.493198 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:15:49.493171 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b\": container with ID starting with e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b not found: ID does not exist" containerID="e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b" Sep 18 21:15:49.493302 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.493212 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b"} err="failed to get container status \"e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b\": rpc error: code = NotFound desc = could not find container \"e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b\": container with ID starting with e1ccd9ff32170d24e052e21dc92a92300126d3c8b1808c0eaac1778cbef66b5b not found: ID does not exist" Sep 18 21:15:49.542096 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.542060 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkwn6\" (UniqueName: \"kubernetes.io/projected/42429754-09cc-4bea-b0ce-c2318081462c-kube-api-access-rkwn6\") pod \"42429754-09cc-4bea-b0ce-c2318081462c\" (UID: \"42429754-09cc-4bea-b0ce-c2318081462c\") " Sep 18 21:15:49.542260 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.542143 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/42429754-09cc-4bea-b0ce-c2318081462c-config-file\") pod \"42429754-09cc-4bea-b0ce-c2318081462c\" (UID: \"42429754-09cc-4bea-b0ce-c2318081462c\") " Sep 18 21:15:49.542517 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.542495 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42429754-09cc-4bea-b0ce-c2318081462c-config-file" (OuterVolumeSpecName: "config-file") pod "42429754-09cc-4bea-b0ce-c2318081462c" (UID: "42429754-09cc-4bea-b0ce-c2318081462c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 21:15:49.544209 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.544184 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42429754-09cc-4bea-b0ce-c2318081462c-kube-api-access-rkwn6" (OuterVolumeSpecName: "kube-api-access-rkwn6") pod "42429754-09cc-4bea-b0ce-c2318081462c" (UID: "42429754-09cc-4bea-b0ce-c2318081462c"). InnerVolumeSpecName "kube-api-access-rkwn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:15:49.643621 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.643583 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkwn6\" (UniqueName: \"kubernetes.io/projected/42429754-09cc-4bea-b0ce-c2318081462c-kube-api-access-rkwn6\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:15:49.643621 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.643616 2557 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/42429754-09cc-4bea-b0ce-c2318081462c-config-file\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:15:49.805449 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.805418 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pwvr4"] Sep 18 21:15:49.811360 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.811330 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pwvr4"] Sep 18 21:15:49.859902 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:49.859867 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42429754-09cc-4bea-b0ce-c2318081462c" path="/var/lib/kubelet/pods/42429754-09cc-4bea-b0ce-c2318081462c/volumes" Sep 18 21:15:55.597888 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.597854 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tx76x"] Sep 18 21:15:55.598246 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.598194 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42429754-09cc-4bea-b0ce-c2318081462c" containerName="limitador" Sep 18 21:15:55.598246 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.598206 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42429754-09cc-4bea-b0ce-c2318081462c" containerName="limitador" Sep 18 21:15:55.598344 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.598270 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42429754-09cc-4bea-b0ce-c2318081462c" containerName="limitador" Sep 18 21:15:55.606628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.606600 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:15:55.612563 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.612538 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 18 21:15:55.612709 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.612637 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 18 21:15:55.613086 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.612797 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 18 21:15:55.613770 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.613750 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8pw68\"" Sep 18 21:15:55.629792 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.629769 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tx76x"] Sep 18 21:15:55.690625 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.690592 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hdc4\" (UniqueName: \"kubernetes.io/projected/0fe4b770-ca6d-43f8-9435-9468258f3f7d-kube-api-access-2hdc4\") pod \"postgres-69fb4594cf-tx76x\" (UID: \"0fe4b770-ca6d-43f8-9435-9468258f3f7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:15:55.690804 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.690634 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0fe4b770-ca6d-43f8-9435-9468258f3f7d-data\") pod \"postgres-69fb4594cf-tx76x\" (UID: \"0fe4b770-ca6d-43f8-9435-9468258f3f7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:15:55.791489 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.791454 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0fe4b770-ca6d-43f8-9435-9468258f3f7d-data\") pod \"postgres-69fb4594cf-tx76x\" (UID: \"0fe4b770-ca6d-43f8-9435-9468258f3f7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:15:55.791674 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.791529 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hdc4\" (UniqueName: \"kubernetes.io/projected/0fe4b770-ca6d-43f8-9435-9468258f3f7d-kube-api-access-2hdc4\") pod \"postgres-69fb4594cf-tx76x\" (UID: \"0fe4b770-ca6d-43f8-9435-9468258f3f7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:15:55.791866 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.791845 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0fe4b770-ca6d-43f8-9435-9468258f3f7d-data\") pod \"postgres-69fb4594cf-tx76x\" (UID: \"0fe4b770-ca6d-43f8-9435-9468258f3f7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:15:55.801598 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.801565 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hdc4\" (UniqueName: \"kubernetes.io/projected/0fe4b770-ca6d-43f8-9435-9468258f3f7d-kube-api-access-2hdc4\") pod \"postgres-69fb4594cf-tx76x\" (UID: \"0fe4b770-ca6d-43f8-9435-9468258f3f7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:15:55.916368 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:55.916305 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:15:56.190948 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:56.190920 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tx76x"] Sep 18 21:15:56.192943 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:15:56.192915 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe4b770_ca6d_43f8_9435_9468258f3f7d.slice/crio-32eeba50680ffea6cb5ce013c8ef2e1d57a690c65dcbb33a0cb9d8cca4ed4822 WatchSource:0}: Error finding container 32eeba50680ffea6cb5ce013c8ef2e1d57a690c65dcbb33a0cb9d8cca4ed4822: Status 404 returned error can't find the container with id 32eeba50680ffea6cb5ce013c8ef2e1d57a690c65dcbb33a0cb9d8cca4ed4822 Sep 18 21:15:56.520850 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:15:56.520764 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" event={"ID":"0fe4b770-ca6d-43f8-9435-9468258f3f7d","Type":"ContainerStarted","Data":"32eeba50680ffea6cb5ce013c8ef2e1d57a690c65dcbb33a0cb9d8cca4ed4822"} Sep 18 21:16:01.312743 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:01.312719 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 18 21:16:01.553507 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:01.553467 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" event={"ID":"0fe4b770-ca6d-43f8-9435-9468258f3f7d","Type":"ContainerStarted","Data":"c9430a52160b9ce97924e4c3c711021c3bf7120f422dc5c23feb6b1321df3102"} Sep 18 21:16:01.553671 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:01.553536 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:16:01.653157 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:01.653104 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" podStartSLOduration=1.53912545 podStartE2EDuration="6.653088905s" totalImagesPullingTime="5.113963455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:15:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:16:01.652855268 +0000 UTC m=+604.301452860" watchObservedRunningTime="2026-09-18 21:16:01.653088905 +0000 UTC m=+604.301686497" Sep 18 21:16:07.588846 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:07.588814 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx76x" Sep 18 21:16:15.572197 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.572155 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-psw2m"] Sep 18 21:16:15.577566 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.577544 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:15.582732 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.582707 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 18 21:16:15.582869 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.582840 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cvq8f\"" Sep 18 21:16:15.583762 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.583744 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 18 21:16:15.583869 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.583747 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 18 21:16:15.607231 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.607202 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-psw2m"] Sep 18 21:16:15.654892 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.654857 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:15.655075 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.654929 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-webhook-cert\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:15.655075 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.655009 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wzwg\" (UniqueName: \"kubernetes.io/projected/fd863065-ca71-45d0-9d39-0760bf332ff6-kube-api-access-6wzwg\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:15.756420 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.756376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-webhook-cert\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:15.756572 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.756455 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wzwg\" (UniqueName: \"kubernetes.io/projected/fd863065-ca71-45d0-9d39-0760bf332ff6-kube-api-access-6wzwg\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:15.756572 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.756493 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:15.756572 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:16:15.756533 2557 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 18 21:16:15.756686 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:16:15.756599 2557 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 18 21:16:15.756686 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:16:15.756617 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-webhook-cert podName:fd863065-ca71-45d0-9d39-0760bf332ff6 nodeName:}" failed. No retries permitted until 2026-09-18 21:16:16.256597832 +0000 UTC m=+618.905195418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-webhook-cert") pod "maas-controller-75dc6cbdf-psw2m" (UID: "fd863065-ca71-45d0-9d39-0760bf332ff6") : secret "maas-controller-webhook-cert" not found Sep 18 21:16:15.756686 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:16:15.756651 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs podName:fd863065-ca71-45d0-9d39-0760bf332ff6 nodeName:}" failed. No retries permitted until 2026-09-18 21:16:16.256634353 +0000 UTC m=+618.905231927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs") pod "maas-controller-75dc6cbdf-psw2m" (UID: "fd863065-ca71-45d0-9d39-0760bf332ff6") : secret "maas-controller-metrics-tls" not found Sep 18 21:16:15.768751 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.768727 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wzwg\" (UniqueName: \"kubernetes.io/projected/fd863065-ca71-45d0-9d39-0760bf332ff6-kube-api-access-6wzwg\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:15.796743 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.796706 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6978ddc9d-gdgmh"] Sep 18 21:16:15.800534 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.800515 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:15.821157 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.821130 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6978ddc9d-gdgmh"] Sep 18 21:16:15.958694 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.958650 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-metrics-certs\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:15.958694 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.958710 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-webhook-cert\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:15.958995 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:15.958805 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxgkh\" (UniqueName: \"kubernetes.io/projected/7f27cf78-b2f3-42fa-8834-6bffa4008f78-kube-api-access-zxgkh\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:16.059635 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.059588 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-metrics-certs\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:16.059635 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.059640 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-webhook-cert\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:16.059897 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.059678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxgkh\" (UniqueName: \"kubernetes.io/projected/7f27cf78-b2f3-42fa-8834-6bffa4008f78-kube-api-access-zxgkh\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:16.059897 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:16:16.059739 2557 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 18 21:16:16.059897 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:16:16.059824 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-metrics-certs podName:7f27cf78-b2f3-42fa-8834-6bffa4008f78 nodeName:}" failed. No retries permitted until 2026-09-18 21:16:16.559804501 +0000 UTC m=+619.208402092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-metrics-certs") pod "maas-controller-6978ddc9d-gdgmh" (UID: "7f27cf78-b2f3-42fa-8834-6bffa4008f78") : secret "maas-controller-metrics-tls" not found Sep 18 21:16:16.062352 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.062303 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-webhook-cert\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:16.070481 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.070453 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxgkh\" (UniqueName: \"kubernetes.io/projected/7f27cf78-b2f3-42fa-8834-6bffa4008f78-kube-api-access-zxgkh\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:16.261541 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.261445 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:16.261691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.261542 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-webhook-cert\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:16.261691 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:16:16.261608 2557 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 18 21:16:16.261784 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:16:16.261702 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs podName:fd863065-ca71-45d0-9d39-0760bf332ff6 nodeName:}" failed. No retries permitted until 2026-09-18 21:16:17.261682902 +0000 UTC m=+619.910280484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs") pod "maas-controller-75dc6cbdf-psw2m" (UID: "fd863065-ca71-45d0-9d39-0760bf332ff6") : secret "maas-controller-metrics-tls" not found Sep 18 21:16:16.263963 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.263933 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-webhook-cert\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:16.564717 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.564629 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-metrics-certs\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:16.567602 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.567578 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-metrics-certs\") pod \"maas-controller-6978ddc9d-gdgmh\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:16.711292 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.711250 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:16.988467 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:16.988442 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6978ddc9d-gdgmh"] Sep 18 21:16:16.990360 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:16:16.990307 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f27cf78_b2f3_42fa_8834_6bffa4008f78.slice/crio-1462b39726cd18bda0d17996d08608d89bba1ec3d448bc9a49d422ff1aa292e3 WatchSource:0}: Error finding container 1462b39726cd18bda0d17996d08608d89bba1ec3d448bc9a49d422ff1aa292e3: Status 404 returned error can't find the container with id 1462b39726cd18bda0d17996d08608d89bba1ec3d448bc9a49d422ff1aa292e3 Sep 18 21:16:17.271724 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:17.271632 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:17.274119 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:17.274088 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs\") pod \"maas-controller-75dc6cbdf-psw2m\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:17.391072 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:17.391031 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:17.638990 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:17.638940 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" event={"ID":"7f27cf78-b2f3-42fa-8834-6bffa4008f78","Type":"ContainerStarted","Data":"1462b39726cd18bda0d17996d08608d89bba1ec3d448bc9a49d422ff1aa292e3"} Sep 18 21:16:17.681684 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:17.681655 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-psw2m"] Sep 18 21:16:17.683782 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:16:17.683751 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd863065_ca71_45d0_9d39_0760bf332ff6.slice/crio-103fb2142f3dca51271aba77ff6e20d2f71dd9476ff932f64da2974b0faba40e WatchSource:0}: Error finding container 103fb2142f3dca51271aba77ff6e20d2f71dd9476ff932f64da2974b0faba40e: Status 404 returned error can't find the container with id 103fb2142f3dca51271aba77ff6e20d2f71dd9476ff932f64da2974b0faba40e Sep 18 21:16:18.647399 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:18.647366 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" event={"ID":"fd863065-ca71-45d0-9d39-0760bf332ff6","Type":"ContainerStarted","Data":"103fb2142f3dca51271aba77ff6e20d2f71dd9476ff932f64da2974b0faba40e"} Sep 18 21:16:19.657077 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:19.657038 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" event={"ID":"7f27cf78-b2f3-42fa-8834-6bffa4008f78","Type":"ContainerStarted","Data":"03c0a1d54510c612574ca1413cba12990d90acfc547b579eb21ccb730823c928"} Sep 18 21:16:19.657534 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:19.657169 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:19.659179 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:19.659148 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" event={"ID":"fd863065-ca71-45d0-9d39-0760bf332ff6","Type":"ContainerStarted","Data":"586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c"} Sep 18 21:16:19.659335 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:19.659272 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:19.682920 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:19.682862 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" podStartSLOduration=2.400853446 podStartE2EDuration="4.682845628s" totalImagesPullingTime="2.281992182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:16:19.680503547 +0000 UTC m=+622.329101138" watchObservedRunningTime="2026-09-18 21:16:19.682845628 +0000 UTC m=+622.331443218" Sep 18 21:16:19.711540 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:19.711485 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" podStartSLOduration=3.12242549 podStartE2EDuration="4.71146895s" totalImagesPullingTime="1.58904346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:16:19.705231918 +0000 UTC m=+622.353829508" watchObservedRunningTime="2026-09-18 21:16:19.71146895 +0000 UTC m=+622.360066541" Sep 18 21:16:30.667045 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.667010 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:16:30.667543 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.667082 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:30.746828 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.746792 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-psw2m"] Sep 18 21:16:30.747034 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.747009 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" podUID="fd863065-ca71-45d0-9d39-0760bf332ff6" containerName="manager" containerID="cri-o://586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c" gracePeriod=30 Sep 18 21:16:30.990670 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.990642 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:30.993744 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.993726 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-webhook-cert\") pod \"fd863065-ca71-45d0-9d39-0760bf332ff6\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " Sep 18 21:16:30.993803 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.993777 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs\") pod \"fd863065-ca71-45d0-9d39-0760bf332ff6\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " Sep 18 21:16:30.993839 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.993807 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wzwg\" (UniqueName: \"kubernetes.io/projected/fd863065-ca71-45d0-9d39-0760bf332ff6-kube-api-access-6wzwg\") pod \"fd863065-ca71-45d0-9d39-0760bf332ff6\" (UID: \"fd863065-ca71-45d0-9d39-0760bf332ff6\") " Sep 18 21:16:30.996044 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.996014 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fd863065-ca71-45d0-9d39-0760bf332ff6" (UID: "fd863065-ca71-45d0-9d39-0760bf332ff6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 21:16:30.996161 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.996065 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fd863065-ca71-45d0-9d39-0760bf332ff6" (UID: "fd863065-ca71-45d0-9d39-0760bf332ff6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 21:16:30.996161 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:30.996084 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd863065-ca71-45d0-9d39-0760bf332ff6-kube-api-access-6wzwg" (OuterVolumeSpecName: "kube-api-access-6wzwg") pod "fd863065-ca71-45d0-9d39-0760bf332ff6" (UID: "fd863065-ca71-45d0-9d39-0760bf332ff6"). InnerVolumeSpecName "kube-api-access-6wzwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:16:31.094751 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.094712 2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-metrics-certs\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:16:31.094751 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.094744 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wzwg\" (UniqueName: \"kubernetes.io/projected/fd863065-ca71-45d0-9d39-0760bf332ff6-kube-api-access-6wzwg\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:16:31.094751 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.094756 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd863065-ca71-45d0-9d39-0760bf332ff6-webhook-cert\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:16:31.722931 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.722892 2557 generic.go:356] "Generic (PLEG): container finished" podID="fd863065-ca71-45d0-9d39-0760bf332ff6" containerID="586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c" exitCode=0 Sep 18 21:16:31.722931 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.722935 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" Sep 18 21:16:31.723484 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.722968 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" event={"ID":"fd863065-ca71-45d0-9d39-0760bf332ff6","Type":"ContainerDied","Data":"586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c"} Sep 18 21:16:31.723484 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.723013 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-psw2m" event={"ID":"fd863065-ca71-45d0-9d39-0760bf332ff6","Type":"ContainerDied","Data":"103fb2142f3dca51271aba77ff6e20d2f71dd9476ff932f64da2974b0faba40e"} Sep 18 21:16:31.723484 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.723037 2557 scope.go:117] "RemoveContainer" containerID="586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c" Sep 18 21:16:31.732140 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.732119 2557 scope.go:117] "RemoveContainer" containerID="586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c" Sep 18 21:16:31.732440 ip-10-0-129-163 kubenswrapper[2557]: E0918 21:16:31.732420 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c\": container with ID starting with 586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c not found: ID does not exist" containerID="586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c" Sep 18 21:16:31.732496 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.732451 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c"} err="failed to get container status \"586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c\": rpc error: code = NotFound desc = could not find container \"586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c\": container with ID starting with 586ac2b44c17d3b8200e5dbdfe90014b99e8070f34d3c416c78b41c3efc7ee2c not found: ID does not exist" Sep 18 21:16:31.746807 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.746778 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-psw2m"] Sep 18 21:16:31.753926 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.753902 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-psw2m"] Sep 18 21:16:31.864327 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:31.864280 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd863065-ca71-45d0-9d39-0760bf332ff6" path="/var/lib/kubelet/pods/fd863065-ca71-45d0-9d39-0760bf332ff6/volumes" Sep 18 21:16:52.650076 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.650031 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz"] Sep 18 21:16:52.652396 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.650534 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd863065-ca71-45d0-9d39-0760bf332ff6" containerName="manager" Sep 18 21:16:52.652396 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.650553 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd863065-ca71-45d0-9d39-0760bf332ff6" containerName="manager" Sep 18 21:16:52.652396 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.650620 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd863065-ca71-45d0-9d39-0760bf332ff6" containerName="manager" Sep 18 21:16:52.653674 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.653658 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.656573 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.656550 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 18 21:16:52.656843 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.656797 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tpgbs\"" Sep 18 21:16:52.660969 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.660945 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 18 21:16:52.668247 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.668225 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz"] Sep 18 21:16:52.769155 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.769124 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-metrics-certs\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.769382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.769162 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-maas-api-tls\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.769382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.769195 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68nsg\" (UniqueName: \"kubernetes.io/projected/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-kube-api-access-68nsg\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.769382 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.769246 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-openshift-service-ca\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.869953 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.869909 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-openshift-service-ca\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.870147 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.869988 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-metrics-certs\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.870147 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.870013 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-maas-api-tls\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.870147 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.870043 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68nsg\" (UniqueName: \"kubernetes.io/projected/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-kube-api-access-68nsg\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.870624 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.870590 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-openshift-service-ca\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.872628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.872605 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-metrics-certs\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.872628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.872623 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-maas-api-tls\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.878023 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.877997 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68nsg\" (UniqueName: \"kubernetes.io/projected/0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92-kube-api-access-68nsg\") pod \"maas-api-59f6566bbc-rv9tz\" (UID: \"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:52.963804 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:52.963722 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:53.173003 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.172967 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6"] Sep 18 21:16:53.176691 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.176668 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:16:53.179728 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.179702 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 18 21:16:53.193152 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.193128 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6"] Sep 18 21:16:53.254596 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.254567 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz"] Sep 18 21:16:53.255239 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:16:53.255204 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f3b2a3e_83c6_40ce_8ebf_ae81d460fe92.slice/crio-9c9249f5fea9d47977ba6d7b4411ef93cb0418710a607293ba97be4e21ff4e86 WatchSource:0}: Error finding container 9c9249f5fea9d47977ba6d7b4411ef93cb0418710a607293ba97be4e21ff4e86: Status 404 returned error can't find the container with id 9c9249f5fea9d47977ba6d7b4411ef93cb0418710a607293ba97be4e21ff4e86 Sep 18 21:16:53.256644 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.256627 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 21:16:53.273367 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.273329 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q98tb\" (UniqueName: \"kubernetes.io/projected/6641a850-a306-4fec-abee-5ce46081aeb0-kube-api-access-q98tb\") pod \"payload-pre-processing-59fbc47bf8-zfwq6\" (UID: \"6641a850-a306-4fec-abee-5ce46081aeb0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:16:53.273502 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.273382 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6641a850-a306-4fec-abee-5ce46081aeb0-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-zfwq6\" (UID: \"6641a850-a306-4fec-abee-5ce46081aeb0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:16:53.374576 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.374538 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6641a850-a306-4fec-abee-5ce46081aeb0-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-zfwq6\" (UID: \"6641a850-a306-4fec-abee-5ce46081aeb0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:16:53.374768 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.374660 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q98tb\" (UniqueName: \"kubernetes.io/projected/6641a850-a306-4fec-abee-5ce46081aeb0-kube-api-access-q98tb\") pod \"payload-pre-processing-59fbc47bf8-zfwq6\" (UID: \"6641a850-a306-4fec-abee-5ce46081aeb0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:16:53.375417 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.375396 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6641a850-a306-4fec-abee-5ce46081aeb0-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-zfwq6\" (UID: \"6641a850-a306-4fec-abee-5ce46081aeb0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:16:53.390264 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.390224 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q98tb\" (UniqueName: \"kubernetes.io/projected/6641a850-a306-4fec-abee-5ce46081aeb0-kube-api-access-q98tb\") pod \"payload-pre-processing-59fbc47bf8-zfwq6\" (UID: \"6641a850-a306-4fec-abee-5ce46081aeb0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:16:53.487083 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.487046 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:16:53.765591 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.765559 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6"] Sep 18 21:16:53.768331 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:16:53.768275 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6641a850_a306_4fec_abee_5ce46081aeb0.slice/crio-812eacc5523f6cf16149b1e79299695f5deca7168a8184686cb8bb2c5aefda63 WatchSource:0}: Error finding container 812eacc5523f6cf16149b1e79299695f5deca7168a8184686cb8bb2c5aefda63: Status 404 returned error can't find the container with id 812eacc5523f6cf16149b1e79299695f5deca7168a8184686cb8bb2c5aefda63 Sep 18 21:16:53.848984 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.848877 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" event={"ID":"6641a850-a306-4fec-abee-5ce46081aeb0","Type":"ContainerStarted","Data":"812eacc5523f6cf16149b1e79299695f5deca7168a8184686cb8bb2c5aefda63"} Sep 18 21:16:53.850375 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:53.850349 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" event={"ID":"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92","Type":"ContainerStarted","Data":"9c9249f5fea9d47977ba6d7b4411ef93cb0418710a607293ba97be4e21ff4e86"} Sep 18 21:16:55.864277 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:55.864237 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" event={"ID":"6641a850-a306-4fec-abee-5ce46081aeb0","Type":"ContainerStarted","Data":"9575412702b5c25084ca38717974d1bbe52427fe611c286c98fedbd831c7f140"} Sep 18 21:16:55.864733 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:55.864300 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:16:55.866085 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:55.866059 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" event={"ID":"0f3b2a3e-83c6-40ce-8ebf-ae81d460fe92","Type":"ContainerStarted","Data":"431fe9b2528052a6ddd207487770f90756c712a0677876f67f644b7551016b36"} Sep 18 21:16:55.866216 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:55.866113 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:16:55.902170 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:55.902118 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" podStartSLOduration=1.560255774 podStartE2EDuration="2.902102459s" totalImagesPullingTime="1.341846685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:16:55.900289975 +0000 UTC m=+658.548887567" watchObservedRunningTime="2026-09-18 21:16:55.902102459 +0000 UTC m=+658.550700084" Sep 18 21:16:55.942011 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:16:55.941960 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" podStartSLOduration=2.088098865 podStartE2EDuration="3.941942469s" totalImagesPullingTime="1.853843604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:16:55.938170719 +0000 UTC m=+658.586768314" watchObservedRunningTime="2026-09-18 21:16:55.941942469 +0000 UTC m=+658.590540060" Sep 18 21:17:01.877278 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:01.877248 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-rv9tz" Sep 18 21:17:06.874401 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:06.874366 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:17:10.824510 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.824471 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj"] Sep 18 21:17:10.829672 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.829649 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:10.833155 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.833128 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 18 21:17:10.833789 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.833770 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 18 21:17:10.834049 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.834030 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9nbxk\"" Sep 18 21:17:10.834125 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.834060 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 18 21:17:10.840281 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.840258 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj"] Sep 18 21:17:10.945518 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.945488 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:10.945518 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.945528 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:10.945786 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.945555 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a416d7-fa3c-4556-974f-6a46f6943d0f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:10.945786 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.945707 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:10.945866 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.945835 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:10.945909 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:10.945863 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d79h9\" (UniqueName: \"kubernetes.io/projected/08a416d7-fa3c-4556-974f-6a46f6943d0f-kube-api-access-d79h9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.047100 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.047066 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.047100 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.047102 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d79h9\" (UniqueName: \"kubernetes.io/projected/08a416d7-fa3c-4556-974f-6a46f6943d0f-kube-api-access-d79h9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.047393 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.047216 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.047393 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.047263 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.047393 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.047288 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a416d7-fa3c-4556-974f-6a46f6943d0f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.047393 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.047376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.047600 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.047562 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.047600 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.047595 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.047710 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.047668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.049900 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.049866 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08a416d7-fa3c-4556-974f-6a46f6943d0f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.050066 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.050043 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08a416d7-fa3c-4556-974f-6a46f6943d0f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.058609 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.058584 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d79h9\" (UniqueName: \"kubernetes.io/projected/08a416d7-fa3c-4556-974f-6a46f6943d0f-kube-api-access-d79h9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj\" (UID: \"08a416d7-fa3c-4556-974f-6a46f6943d0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.140985 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.140949 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:11.417043 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.417016 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj"] Sep 18 21:17:11.418795 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:17:11.418765 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a416d7_fa3c_4556_974f_6a46f6943d0f.slice/crio-c7e9adff4e7232e5966e28987a730be346d4d102552d6370ba8545f93a2114af WatchSource:0}: Error finding container c7e9adff4e7232e5966e28987a730be346d4d102552d6370ba8545f93a2114af: Status 404 returned error can't find the container with id c7e9adff4e7232e5966e28987a730be346d4d102552d6370ba8545f93a2114af Sep 18 21:17:11.956817 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:11.956775 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" event={"ID":"08a416d7-fa3c-4556-974f-6a46f6943d0f","Type":"ContainerStarted","Data":"c7e9adff4e7232e5966e28987a730be346d4d102552d6370ba8545f93a2114af"} Sep 18 21:17:12.964052 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:12.964021 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" event={"ID":"08a416d7-fa3c-4556-974f-6a46f6943d0f","Type":"ContainerStarted","Data":"3908c0964a075bd29dbc073a409b52dab4addd68215872ddd4253bd140606c36"} Sep 18 21:17:12.986001 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:12.985946 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" podStartSLOduration=1.9837683940000002 podStartE2EDuration="2.985930201s" totalImagesPullingTime="1.002161807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:17:12.98263285 +0000 UTC m=+675.631230441" watchObservedRunningTime="2026-09-18 21:17:12.985930201 +0000 UTC m=+675.634527793" Sep 18 21:17:21.141293 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:21.141251 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:21.141719 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:21.141310 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:21.153754 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:21.153727 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:22.020766 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:22.020735 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9x5zj" Sep 18 21:17:51.322525 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.322442 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm"] Sep 18 21:17:51.325497 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.325479 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.328223 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.328199 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 18 21:17:51.337159 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.337131 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm"] Sep 18 21:17:51.399867 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.399822 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.399867 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.399862 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.400115 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.399892 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.400115 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.399964 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.400115 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.400019 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.400115 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.400051 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpt8b\" (UniqueName: \"kubernetes.io/projected/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-kube-api-access-mpt8b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.501082 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.501043 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.501274 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.501090 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpt8b\" (UniqueName: \"kubernetes.io/projected/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-kube-api-access-mpt8b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.501274 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.501128 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.501274 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.501146 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.501274 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.501176 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.501274 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.501197 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.501623 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.501599 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.501685 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.501633 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.501736 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.501677 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.503623 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.503599 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.503845 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.503826 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.517186 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.517156 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpt8b\" (UniqueName: \"kubernetes.io/projected/fa228d68-13ba-4cd7-8bc5-2a505ee7406c-kube-api-access-mpt8b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm\" (UID: \"fa228d68-13ba-4cd7-8bc5-2a505ee7406c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.636700 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.636668 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:17:51.919097 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:51.919069 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm"] Sep 18 21:17:51.920196 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:17:51.920157 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa228d68_13ba_4cd7_8bc5_2a505ee7406c.slice/crio-b2593ce9432480150b33dadb112e63143acd6f895019389e6bad5e241ecff01f WatchSource:0}: Error finding container b2593ce9432480150b33dadb112e63143acd6f895019389e6bad5e241ecff01f: Status 404 returned error can't find the container with id b2593ce9432480150b33dadb112e63143acd6f895019389e6bad5e241ecff01f Sep 18 21:17:52.170461 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:52.170377 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" event={"ID":"fa228d68-13ba-4cd7-8bc5-2a505ee7406c","Type":"ContainerStarted","Data":"56e5a4ad1e3bde728dc9ce1f8c2ceaa3ef51bb1a5cec2f7c45fab3056a8ef338"} Sep 18 21:17:52.170461 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:52.170430 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" event={"ID":"fa228d68-13ba-4cd7-8bc5-2a505ee7406c","Type":"ContainerStarted","Data":"b2593ce9432480150b33dadb112e63143acd6f895019389e6bad5e241ecff01f"} Sep 18 21:17:52.194892 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:17:52.194838 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" podStartSLOduration=1.030650679 podStartE2EDuration="1.194820973s" totalImagesPullingTime="164.170294ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:17:52.190007903 +0000 UTC m=+714.838605494" watchObservedRunningTime="2026-09-18 21:17:52.194820973 +0000 UTC m=+714.843418564" Sep 18 21:18:01.636955 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:01.636905 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:18:01.637349 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:01.636978 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:18:01.649602 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:01.649574 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:18:02.232068 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:02.232034 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvhzm" Sep 18 21:18:18.200860 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.200825 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4"] Sep 18 21:18:18.203357 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.203337 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.206851 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.206821 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 18 21:18:18.218628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.218605 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4"] Sep 18 21:18:18.234348 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.234296 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp995\" (UniqueName: \"kubernetes.io/projected/b75475dc-49d4-4c62-89cd-8d6ea1bae574-kube-api-access-tp995\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.234476 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.234414 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.234476 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.234444 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b75475dc-49d4-4c62-89cd-8d6ea1bae574-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.234476 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.234474 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.234593 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.234491 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.234628 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.234600 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.335542 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.335504 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.335710 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.335550 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp995\" (UniqueName: \"kubernetes.io/projected/b75475dc-49d4-4c62-89cd-8d6ea1bae574-kube-api-access-tp995\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.335710 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.335603 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.335710 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.335618 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b75475dc-49d4-4c62-89cd-8d6ea1bae574-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.335710 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.335641 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.335710 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.335657 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.336028 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.335994 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.336179 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.336064 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.336179 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.336066 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.337995 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.337969 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b75475dc-49d4-4c62-89cd-8d6ea1bae574-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.338243 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.338220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b75475dc-49d4-4c62-89cd-8d6ea1bae574-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.344751 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.344724 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp995\" (UniqueName: \"kubernetes.io/projected/b75475dc-49d4-4c62-89cd-8d6ea1bae574-kube-api-access-tp995\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t5vq4\" (UID: \"b75475dc-49d4-4c62-89cd-8d6ea1bae574\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.513545 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.513449 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:18.787102 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:18.787072 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4"] Sep 18 21:18:18.789112 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:18:18.789079 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb75475dc_49d4_4c62_89cd_8d6ea1bae574.slice/crio-22f0782f2647661c8c7a13170d51078d5e050c93cfb1a034713b59f2539a9d12 WatchSource:0}: Error finding container 22f0782f2647661c8c7a13170d51078d5e050c93cfb1a034713b59f2539a9d12: Status 404 returned error can't find the container with id 22f0782f2647661c8c7a13170d51078d5e050c93cfb1a034713b59f2539a9d12 Sep 18 21:18:19.327964 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:19.327925 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" event={"ID":"b75475dc-49d4-4c62-89cd-8d6ea1bae574","Type":"ContainerStarted","Data":"9c0a2e61af54d66c74ec29f6db01da3f2ed41f595845e99a261fc9b705fe6662"} Sep 18 21:18:19.328396 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:19.328006 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" event={"ID":"b75475dc-49d4-4c62-89cd-8d6ea1bae574","Type":"ContainerStarted","Data":"22f0782f2647661c8c7a13170d51078d5e050c93cfb1a034713b59f2539a9d12"} Sep 18 21:18:28.513802 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:28.513764 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:28.514276 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:28.513839 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:28.526366 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:28.526339 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:18:28.552889 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:28.552837 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" podStartSLOduration=10.384065205 podStartE2EDuration="10.552820694s" totalImagesPullingTime="168.755489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:18:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:18:19.372915699 +0000 UTC m=+742.021513292" watchObservedRunningTime="2026-09-18 21:18:28.552820694 +0000 UTC m=+751.201418330" Sep 18 21:18:29.393452 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:18:29.393419 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t5vq4" Sep 18 21:19:54.576668 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.576631 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6"] Sep 18 21:19:54.577196 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.576885 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" podUID="6641a850-a306-4fec-abee-5ce46081aeb0" containerName="payload-pre-processing" containerID="cri-o://9575412702b5c25084ca38717974d1bbe52427fe611c286c98fedbd831c7f140" gracePeriod=30 Sep 18 21:19:54.631260 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.631218 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6978ddc9d-gdgmh"] Sep 18 21:19:54.631536 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.631493 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" podUID="7f27cf78-b2f3-42fa-8834-6bffa4008f78" containerName="manager" containerID="cri-o://03c0a1d54510c612574ca1413cba12990d90acfc547b579eb21ccb730823c928" gracePeriod=30 Sep 18 21:19:54.845937 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.845803 2557 generic.go:356] "Generic (PLEG): container finished" podID="7f27cf78-b2f3-42fa-8834-6bffa4008f78" containerID="03c0a1d54510c612574ca1413cba12990d90acfc547b579eb21ccb730823c928" exitCode=0 Sep 18 21:19:54.845937 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.845882 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" event={"ID":"7f27cf78-b2f3-42fa-8834-6bffa4008f78","Type":"ContainerDied","Data":"03c0a1d54510c612574ca1413cba12990d90acfc547b579eb21ccb730823c928"} Sep 18 21:19:54.848029 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.848001 2557 generic.go:356] "Generic (PLEG): container finished" podID="6641a850-a306-4fec-abee-5ce46081aeb0" containerID="9575412702b5c25084ca38717974d1bbe52427fe611c286c98fedbd831c7f140" exitCode=0 Sep 18 21:19:54.848129 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.848079 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" event={"ID":"6641a850-a306-4fec-abee-5ce46081aeb0","Type":"ContainerDied","Data":"9575412702b5c25084ca38717974d1bbe52427fe611c286c98fedbd831c7f140"} Sep 18 21:19:54.876737 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.876715 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:19:54.883004 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.882985 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6641a850-a306-4fec-abee-5ce46081aeb0-plugins-config-volume\") pod \"6641a850-a306-4fec-abee-5ce46081aeb0\" (UID: \"6641a850-a306-4fec-abee-5ce46081aeb0\") " Sep 18 21:19:54.883091 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.883019 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q98tb\" (UniqueName: \"kubernetes.io/projected/6641a850-a306-4fec-abee-5ce46081aeb0-kube-api-access-q98tb\") pod \"6641a850-a306-4fec-abee-5ce46081aeb0\" (UID: \"6641a850-a306-4fec-abee-5ce46081aeb0\") " Sep 18 21:19:54.883396 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.883373 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6641a850-a306-4fec-abee-5ce46081aeb0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6641a850-a306-4fec-abee-5ce46081aeb0" (UID: "6641a850-a306-4fec-abee-5ce46081aeb0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 21:19:54.885092 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.885071 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6641a850-a306-4fec-abee-5ce46081aeb0-kube-api-access-q98tb" (OuterVolumeSpecName: "kube-api-access-q98tb") pod "6641a850-a306-4fec-abee-5ce46081aeb0" (UID: "6641a850-a306-4fec-abee-5ce46081aeb0"). InnerVolumeSpecName "kube-api-access-q98tb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:19:54.899084 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.899064 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:19:54.983872 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.983841 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxgkh\" (UniqueName: \"kubernetes.io/projected/7f27cf78-b2f3-42fa-8834-6bffa4008f78-kube-api-access-zxgkh\") pod \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " Sep 18 21:19:54.984051 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.983884 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-webhook-cert\") pod \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " Sep 18 21:19:54.984051 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.983942 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-metrics-certs\") pod \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\" (UID: \"7f27cf78-b2f3-42fa-8834-6bffa4008f78\") " Sep 18 21:19:54.984239 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.984219 2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6641a850-a306-4fec-abee-5ce46081aeb0-plugins-config-volume\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:19:54.984348 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.984243 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q98tb\" (UniqueName: \"kubernetes.io/projected/6641a850-a306-4fec-abee-5ce46081aeb0-kube-api-access-q98tb\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:19:54.986087 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.986060 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7f27cf78-b2f3-42fa-8834-6bffa4008f78" (UID: "7f27cf78-b2f3-42fa-8834-6bffa4008f78"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 21:19:54.986188 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.986092 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7f27cf78-b2f3-42fa-8834-6bffa4008f78" (UID: "7f27cf78-b2f3-42fa-8834-6bffa4008f78"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 21:19:54.986288 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:54.986269 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f27cf78-b2f3-42fa-8834-6bffa4008f78-kube-api-access-zxgkh" (OuterVolumeSpecName: "kube-api-access-zxgkh") pod "7f27cf78-b2f3-42fa-8834-6bffa4008f78" (UID: "7f27cf78-b2f3-42fa-8834-6bffa4008f78"). InnerVolumeSpecName "kube-api-access-zxgkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 21:19:55.085518 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.085484 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-webhook-cert\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:19:55.085518 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.085511 2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f27cf78-b2f3-42fa-8834-6bffa4008f78-metrics-certs\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:19:55.085518 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.085522 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxgkh\" (UniqueName: \"kubernetes.io/projected/7f27cf78-b2f3-42fa-8834-6bffa4008f78-kube-api-access-zxgkh\") on node \"ip-10-0-129-163.ec2.internal\" DevicePath \"\"" Sep 18 21:19:55.855984 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.855952 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" Sep 18 21:19:55.857444 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.857419 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" Sep 18 21:19:55.861146 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.861115 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6978ddc9d-gdgmh" event={"ID":"7f27cf78-b2f3-42fa-8834-6bffa4008f78","Type":"ContainerDied","Data":"1462b39726cd18bda0d17996d08608d89bba1ec3d448bc9a49d422ff1aa292e3"} Sep 18 21:19:55.861280 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.861175 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6" event={"ID":"6641a850-a306-4fec-abee-5ce46081aeb0","Type":"ContainerDied","Data":"812eacc5523f6cf16149b1e79299695f5deca7168a8184686cb8bb2c5aefda63"} Sep 18 21:19:55.861280 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.861213 2557 scope.go:117] "RemoveContainer" containerID="03c0a1d54510c612574ca1413cba12990d90acfc547b579eb21ccb730823c928" Sep 18 21:19:55.872690 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.872671 2557 scope.go:117] "RemoveContainer" containerID="9575412702b5c25084ca38717974d1bbe52427fe611c286c98fedbd831c7f140" Sep 18 21:19:55.908494 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.908463 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6978ddc9d-gdgmh"] Sep 18 21:19:55.924551 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.924518 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6978ddc9d-gdgmh"] Sep 18 21:19:55.940904 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.940879 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6"] Sep 18 21:19:55.949015 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.948991 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-zfwq6"] Sep 18 21:19:55.980897 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.980861 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6978ddc9d-cbp2s"] Sep 18 21:19:55.981281 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.981263 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f27cf78-b2f3-42fa-8834-6bffa4008f78" containerName="manager" Sep 18 21:19:55.981387 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.981283 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f27cf78-b2f3-42fa-8834-6bffa4008f78" containerName="manager" Sep 18 21:19:55.981387 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.981304 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6641a850-a306-4fec-abee-5ce46081aeb0" containerName="payload-pre-processing" Sep 18 21:19:55.981387 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.981327 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6641a850-a306-4fec-abee-5ce46081aeb0" containerName="payload-pre-processing" Sep 18 21:19:55.981553 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.981414 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6641a850-a306-4fec-abee-5ce46081aeb0" containerName="payload-pre-processing" Sep 18 21:19:55.981553 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.981428 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f27cf78-b2f3-42fa-8834-6bffa4008f78" containerName="manager" Sep 18 21:19:55.985593 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.985574 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:55.992351 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.992301 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 18 21:19:55.992490 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.992387 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 18 21:19:55.992637 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.992621 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 18 21:19:55.994919 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.994897 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg7kh\" (UniqueName: \"kubernetes.io/projected/28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce-kube-api-access-gg7kh\") pod \"maas-controller-6978ddc9d-cbp2s\" (UID: \"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce\") " pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:55.995016 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.994930 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce-metrics-certs\") pod \"maas-controller-6978ddc9d-cbp2s\" (UID: \"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce\") " pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:55.995072 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:55.995017 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce-webhook-cert\") pod \"maas-controller-6978ddc9d-cbp2s\" (UID: \"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce\") " pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:56.002020 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.002003 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cvq8f\"" Sep 18 21:19:56.036872 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.036836 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6978ddc9d-cbp2s"] Sep 18 21:19:56.095643 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.095610 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg7kh\" (UniqueName: \"kubernetes.io/projected/28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce-kube-api-access-gg7kh\") pod \"maas-controller-6978ddc9d-cbp2s\" (UID: \"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce\") " pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:56.095643 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.095648 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce-metrics-certs\") pod \"maas-controller-6978ddc9d-cbp2s\" (UID: \"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce\") " pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:56.095856 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.095697 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce-webhook-cert\") pod \"maas-controller-6978ddc9d-cbp2s\" (UID: \"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce\") " pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:56.098228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.098205 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce-webhook-cert\") pod \"maas-controller-6978ddc9d-cbp2s\" (UID: \"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce\") " pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:56.098228 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.098220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce-metrics-certs\") pod \"maas-controller-6978ddc9d-cbp2s\" (UID: \"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce\") " pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:56.120919 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.120855 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg7kh\" (UniqueName: \"kubernetes.io/projected/28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce-kube-api-access-gg7kh\") pod \"maas-controller-6978ddc9d-cbp2s\" (UID: \"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce\") " pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:56.296184 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.296145 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:56.595107 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.595080 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6978ddc9d-cbp2s"] Sep 18 21:19:56.596867 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:19:56.596839 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28942c83_2c64_4eed_8c9f_2ac2c9a4e2ce.slice/crio-de5520f93e0c639d368cb6e400e44fc0f336cafdaa5d2513f90dbf020c3ad94c WatchSource:0}: Error finding container de5520f93e0c639d368cb6e400e44fc0f336cafdaa5d2513f90dbf020c3ad94c: Status 404 returned error can't find the container with id de5520f93e0c639d368cb6e400e44fc0f336cafdaa5d2513f90dbf020c3ad94c Sep 18 21:19:56.865992 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:56.865952 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6978ddc9d-cbp2s" event={"ID":"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce","Type":"ContainerStarted","Data":"de5520f93e0c639d368cb6e400e44fc0f336cafdaa5d2513f90dbf020c3ad94c"} Sep 18 21:19:57.863089 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:57.863053 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6641a850-a306-4fec-abee-5ce46081aeb0" path="/var/lib/kubelet/pods/6641a850-a306-4fec-abee-5ce46081aeb0/volumes" Sep 18 21:19:57.863600 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:57.863583 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f27cf78-b2f3-42fa-8834-6bffa4008f78" path="/var/lib/kubelet/pods/7f27cf78-b2f3-42fa-8834-6bffa4008f78/volumes" Sep 18 21:19:57.878258 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:57.878226 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6978ddc9d-cbp2s" event={"ID":"28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce","Type":"ContainerStarted","Data":"69a91cf8d7b3e981efb03bd497873aca23d5c866f8ed9bdba2660e66c82519ea"} Sep 18 21:19:57.878623 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:57.878392 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:19:57.914146 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:19:57.914091 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6978ddc9d-cbp2s" podStartSLOduration=2.496588962 podStartE2EDuration="2.914073845s" totalImagesPullingTime="417.484883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:19:57.910972366 +0000 UTC m=+840.559569957" watchObservedRunningTime="2026-09-18 21:19:57.914073845 +0000 UTC m=+840.562671457" Sep 18 21:20:08.886084 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:20:08.886051 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6978ddc9d-cbp2s" Sep 18 21:27:42.875639 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:42.875599 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t8gmh/must-gather-7s5vq"] Sep 18 21:27:42.879305 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:42.879287 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t8gmh/must-gather-7s5vq" Sep 18 21:27:42.882188 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:42.882166 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t8gmh\"/\"kube-root-ca.crt\"" Sep 18 21:27:42.883124 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:42.883100 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t8gmh\"/\"default-dockercfg-7lsc7\"" Sep 18 21:27:42.883124 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:42.883119 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t8gmh\"/\"openshift-service-ca.crt\"" Sep 18 21:27:42.958854 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:42.958814 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t8gmh/must-gather-7s5vq"] Sep 18 21:27:43.010832 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:43.010785 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f64fd28b-8234-4aa3-be3b-1ed055c5a881-must-gather-output\") pod \"must-gather-7s5vq\" (UID: \"f64fd28b-8234-4aa3-be3b-1ed055c5a881\") " pod="openshift-must-gather-t8gmh/must-gather-7s5vq" Sep 18 21:27:43.011012 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:43.010892 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ndtw\" (UniqueName: \"kubernetes.io/projected/f64fd28b-8234-4aa3-be3b-1ed055c5a881-kube-api-access-4ndtw\") pod \"must-gather-7s5vq\" (UID: \"f64fd28b-8234-4aa3-be3b-1ed055c5a881\") " pod="openshift-must-gather-t8gmh/must-gather-7s5vq" Sep 18 21:27:43.111687 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:43.111642 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ndtw\" (UniqueName: \"kubernetes.io/projected/f64fd28b-8234-4aa3-be3b-1ed055c5a881-kube-api-access-4ndtw\") pod \"must-gather-7s5vq\" (UID: \"f64fd28b-8234-4aa3-be3b-1ed055c5a881\") " pod="openshift-must-gather-t8gmh/must-gather-7s5vq" Sep 18 21:27:43.111878 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:43.111720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f64fd28b-8234-4aa3-be3b-1ed055c5a881-must-gather-output\") pod \"must-gather-7s5vq\" (UID: \"f64fd28b-8234-4aa3-be3b-1ed055c5a881\") " pod="openshift-must-gather-t8gmh/must-gather-7s5vq" Sep 18 21:27:43.112053 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:43.112037 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f64fd28b-8234-4aa3-be3b-1ed055c5a881-must-gather-output\") pod \"must-gather-7s5vq\" (UID: \"f64fd28b-8234-4aa3-be3b-1ed055c5a881\") " pod="openshift-must-gather-t8gmh/must-gather-7s5vq" Sep 18 21:27:43.121038 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:43.121013 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ndtw\" (UniqueName: \"kubernetes.io/projected/f64fd28b-8234-4aa3-be3b-1ed055c5a881-kube-api-access-4ndtw\") pod \"must-gather-7s5vq\" (UID: \"f64fd28b-8234-4aa3-be3b-1ed055c5a881\") " pod="openshift-must-gather-t8gmh/must-gather-7s5vq" Sep 18 21:27:43.192465 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:43.192377 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t8gmh/must-gather-7s5vq" Sep 18 21:27:43.491423 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:43.491395 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t8gmh/must-gather-7s5vq"] Sep 18 21:27:43.493483 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:27:43.493451 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf64fd28b_8234_4aa3_be3b_1ed055c5a881.slice/crio-9e494203892b2286c1cb111b02bbdaefd678a15723facd5e449480d78c8a3ad8 WatchSource:0}: Error finding container 9e494203892b2286c1cb111b02bbdaefd678a15723facd5e449480d78c8a3ad8: Status 404 returned error can't find the container with id 9e494203892b2286c1cb111b02bbdaefd678a15723facd5e449480d78c8a3ad8 Sep 18 21:27:43.495408 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:43.495387 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 21:27:44.368401 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:44.368371 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t8gmh/must-gather-7s5vq" event={"ID":"f64fd28b-8234-4aa3-be3b-1ed055c5a881","Type":"ContainerStarted","Data":"9e494203892b2286c1cb111b02bbdaefd678a15723facd5e449480d78c8a3ad8"} Sep 18 21:27:45.378522 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:45.378488 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t8gmh/must-gather-7s5vq" event={"ID":"f64fd28b-8234-4aa3-be3b-1ed055c5a881","Type":"ContainerStarted","Data":"f42b08bfbf7226a7ac7caeb737ecdaf02693f1cb5f2e17c2d6d3fa0ae78d230c"} Sep 18 21:27:45.378907 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:45.378532 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t8gmh/must-gather-7s5vq" event={"ID":"f64fd28b-8234-4aa3-be3b-1ed055c5a881","Type":"ContainerStarted","Data":"85d7cf7d9d6fb0dce57891fc8f0bcf87dccbb95f97186753cd0d2ecb5faab40e"} Sep 18 21:27:45.398433 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:45.398377 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t8gmh/must-gather-7s5vq" podStartSLOduration=2.585145766 podStartE2EDuration="3.398358213s" totalImagesPullingTime="813.212447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 21:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:27:45.395992906 +0000 UTC m=+1308.044590525" watchObservedRunningTime="2026-09-18 21:27:45.398358213 +0000 UTC m=+1308.046955795" Sep 18 21:27:46.090616 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:46.090587 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hrxgz_cbba6619-7498-4ef7-a9a8-174ffc7c3953/global-pull-secret-syncer/0.log" Sep 18 21:27:46.189448 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:46.189399 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jhzsp_369e25ea-147c-4099-a089-03876f8f9ec5/konnectivity-agent/0.log" Sep 18 21:27:46.250250 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:46.250184 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-163.ec2.internal_0177e4b7799939fec863354aec680a58/haproxy/0.log" Sep 18 21:27:50.245954 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:50.245925 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-6f9dn_d4886633-1e3c-48b0-8beb-6d8cd01d413d/manager/0.log" Sep 18 21:27:50.299934 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:50.299878 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-gzqzc_23c92796-0e77-4002-ab26-cf79366e832f/manager/0.log" Sep 18 21:27:50.334288 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:50.334192 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-ql4zl_9442400d-1fde-4963-bddf-d1bc61c9e466/kuadrant-console-plugin/0.log" Sep 18 21:27:50.380712 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:50.380642 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-7hzcj_aa15d486-0ac7-4b4a-b9d4-c7796d2ae2fa/manager/0.log" Sep 18 21:27:50.447622 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:50.447571 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-td8fr_42c966c3-eb8f-4dc9-b481-35e977252077/manager/0.log" Sep 18 21:27:52.737545 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:52.737512 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-89cb5fbc4-svcnv_31cbff63-8b3a-420d-a9b3-9310f05cbcd3/metrics-server/0.log" Sep 18 21:27:52.922698 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:52.922660 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dps6z_3382c007-d5c3-4dc8-8e6c-994884dbb284/node-exporter/0.log" Sep 18 21:27:52.950772 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:52.950742 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dps6z_3382c007-d5c3-4dc8-8e6c-994884dbb284/kube-rbac-proxy/0.log" Sep 18 21:27:52.982342 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:52.982297 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dps6z_3382c007-d5c3-4dc8-8e6c-994884dbb284/init-textfile/0.log" Sep 18 21:27:55.179738 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:55.179704 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-hbflc_c99ba6d4-0611-419c-87e4-e927682a92b8/networking-console-plugin/0.log" Sep 18 21:27:56.089653 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.089623 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt"] Sep 18 21:27:56.095732 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.095712 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.126360 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.126305 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt"] Sep 18 21:27:56.247526 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.247487 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-sys\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.247919 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.247555 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-proc\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.247919 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.247597 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-lib-modules\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.247919 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.247616 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnfvj\" (UniqueName: \"kubernetes.io/projected/7c26f128-bb0a-44f1-9466-6e1791c016a7-kube-api-access-gnfvj\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.247919 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.247687 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-podres\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.348500 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.348417 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-proc\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.348500 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.348464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-lib-modules\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.348500 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.348495 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnfvj\" (UniqueName: \"kubernetes.io/projected/7c26f128-bb0a-44f1-9466-6e1791c016a7-kube-api-access-gnfvj\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.348735 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.348552 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-proc\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.348735 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.348561 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-podres\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.348735 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.348660 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-lib-modules\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.348735 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.348668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-podres\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.348735 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.348697 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-sys\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.348924 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.348790 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c26f128-bb0a-44f1-9466-6e1791c016a7-sys\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.368331 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.368287 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnfvj\" (UniqueName: \"kubernetes.io/projected/7c26f128-bb0a-44f1-9466-6e1791c016a7-kube-api-access-gnfvj\") pod \"perf-node-gather-daemonset-8shxt\" (UID: \"7c26f128-bb0a-44f1-9466-6e1791c016a7\") " pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.407457 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.407421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:56.730695 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:56.730661 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt"] Sep 18 21:27:56.733128 ip-10-0-129-163 kubenswrapper[2557]: W0918 21:27:56.733097 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7c26f128_bb0a_44f1_9466_6e1791c016a7.slice/crio-709b55921b749b142c7bb0109f8518fc741f9f9dda3032a5da14b8679035b1d2 WatchSource:0}: Error finding container 709b55921b749b142c7bb0109f8518fc741f9f9dda3032a5da14b8679035b1d2: Status 404 returned error can't find the container with id 709b55921b749b142c7bb0109f8518fc741f9f9dda3032a5da14b8679035b1d2 Sep 18 21:27:57.477832 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:57.477167 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" event={"ID":"7c26f128-bb0a-44f1-9466-6e1791c016a7","Type":"ContainerStarted","Data":"4ccb1103fc9f188ec9cbcb293ce49c1c4bba09cd40a7f9079d465a17eee725f8"} Sep 18 21:27:57.478333 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:57.477877 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" event={"ID":"7c26f128-bb0a-44f1-9466-6e1791c016a7","Type":"ContainerStarted","Data":"709b55921b749b142c7bb0109f8518fc741f9f9dda3032a5da14b8679035b1d2"} Sep 18 21:27:57.478333 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:57.477908 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:27:57.506176 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:57.506117 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" podStartSLOduration=1.506100317 podStartE2EDuration="1.506100317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 21:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 21:27:57.50589376 +0000 UTC m=+1320.154491352" watchObservedRunningTime="2026-09-18 21:27:57.506100317 +0000 UTC m=+1320.154697909" Sep 18 21:27:59.015679 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:59.015649 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tsxlb_2321d814-a682-4e63-bc8b-c174d6cea7bc/dns/0.log" Sep 18 21:27:59.043138 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:59.043087 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tsxlb_2321d814-a682-4e63-bc8b-c174d6cea7bc/kube-rbac-proxy/0.log" Sep 18 21:27:59.135849 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:59.135808 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pj68m_626b4d06-d3b0-44fb-94b9-94f3131d6afc/dns-node-resolver/0.log" Sep 18 21:27:59.907983 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:27:59.907940 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pvrc4_28264d96-1e7c-49bf-9654-5ad9c675d9b6/node-ca/0.log" Sep 18 21:28:01.144515 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:01.144487 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bf5bc8c8c-kss4j_2ed9db4a-726e-4f14-84e6-c27bf160c765/kube-auth-proxy/0.log" Sep 18 21:28:01.841092 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:01.841058 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5n7pb_37a4721d-1c7f-434e-97d0-f1d32baae843/serve-healthcheck-canary/0.log" Sep 18 21:28:02.779759 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:02.779728 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mhgzm_22d79160-d48c-4229-9681-e10eefd24c2b/kube-rbac-proxy/0.log" Sep 18 21:28:02.809450 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:02.809420 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mhgzm_22d79160-d48c-4229-9681-e10eefd24c2b/exporter/0.log" Sep 18 21:28:02.842968 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:02.842942 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mhgzm_22d79160-d48c-4229-9681-e10eefd24c2b/extractor/0.log" Sep 18 21:28:04.502720 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:04.502692 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t8gmh/perf-node-gather-daemonset-8shxt" Sep 18 21:28:05.256377 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:05.256349 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-62bxk_3dfa38bb-1d1b-48b0-8fc2-bf11e8702427/manager/0.log" Sep 18 21:28:05.303789 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:05.303759 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ns59x_05762543-5954-4a6d-975e-64266f00638b/manager/0.log" Sep 18 21:28:05.344831 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:05.344760 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6978ddc9d-cbp2s_28942c83-2c64-4eed-8c9f-2ac2c9a4e2ce/manager/0.log" Sep 18 21:28:05.692486 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:05.692457 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp_de53b195-a3ff-4a12-aa70-a1395c1fc918/manager/0.log" Sep 18 21:28:05.732265 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:05.732235 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc5c55ffb-zxmnp_de53b195-a3ff-4a12-aa70-a1395c1fc918/copy-manifests/0.log" Sep 18 21:28:06.785564 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:06.785441 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bf5bc8c8c-kss4j_2ed9db4a-726e-4f14-84e6-c27bf160c765/kube-auth-proxy/0.log" Sep 18 21:28:08.247813 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:08.247768 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bf5bc8c8c-kss4j_2ed9db4a-726e-4f14-84e6-c27bf160c765/kube-auth-proxy/0.log" Sep 18 21:28:14.607600 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:14.607568 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2pc68_535650fe-fb2f-4b86-9c22-23eec74be1a1/kube-multus/0.log" Sep 18 21:28:14.916378 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:14.916294 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9sm5k_597032d6-55fc-4ec1-a791-533a72693ad2/kube-multus-additional-cni-plugins/0.log" Sep 18 21:28:14.943352 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:14.943300 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9sm5k_597032d6-55fc-4ec1-a791-533a72693ad2/egress-router-binary-copy/0.log" Sep 18 21:28:14.975761 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:14.975733 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9sm5k_597032d6-55fc-4ec1-a791-533a72693ad2/cni-plugins/0.log" Sep 18 21:28:15.012372 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:15.012342 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9sm5k_597032d6-55fc-4ec1-a791-533a72693ad2/bond-cni-plugin/0.log" Sep 18 21:28:15.042857 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:15.042817 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9sm5k_597032d6-55fc-4ec1-a791-533a72693ad2/routeoverride-cni/0.log" Sep 18 21:28:15.069851 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:15.069800 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9sm5k_597032d6-55fc-4ec1-a791-533a72693ad2/whereabouts-cni-bincopy/0.log" Sep 18 21:28:15.109995 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:15.109969 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9sm5k_597032d6-55fc-4ec1-a791-533a72693ad2/whereabouts-cni/0.log" Sep 18 21:28:15.341343 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:15.341249 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bvvml_3ff122f5-c7e3-4cc6-b2a7-46b532f84d50/network-metrics-daemon/0.log" Sep 18 21:28:15.365130 ip-10-0-129-163 kubenswrapper[2557]: I0918 21:28:15.365100 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bvvml_3ff122f5-c7e3-4cc6-b2a7-46b532f84d50/kube-rbac-proxy/0.log"