Sep 16 07:27:46.768885 ip-10-0-136-65 systemd[1]: Starting Kubernetes Kubelet... Sep 16 07:27:47.110955 ip-10-0-136-65 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 16 07:27:47.110955 ip-10-0-136-65 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 16 07:27:47.110955 ip-10-0-136-65 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 16 07:27:47.110955 ip-10-0-136-65 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 16 07:27:47.110955 ip-10-0-136-65 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 16 07:27:47.113218 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.113137 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 16 07:27:47.118244 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118225 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 07:27:47.118244 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118242 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 07:27:47.118244 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118247 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118251 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118255 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118259 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118263 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118268 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118272 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118275 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118279 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118283 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118287 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118293 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118299 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118305 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118309 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118314 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118318 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118327 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118331 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 07:27:47.118453 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118335 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118339 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118343 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118347 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118352 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118356 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118360 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118364 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118369 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118373 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118377 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118385 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118391 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118404 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118408 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118413 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118419 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118425 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118429 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 07:27:47.119253 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118433 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118438 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118442 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118446 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118451 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118455 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118459 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118463 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118467 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118472 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118476 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118480 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118484 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118488 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118493 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118497 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118501 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118506 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118510 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118514 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 07:27:47.120057 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118518 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118523 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118527 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118531 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118535 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118540 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118544 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118548 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118554 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118558 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118563 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118568 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118573 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118577 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118583 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118587 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118591 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118595 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118599 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118604 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 07:27:47.120903 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118608 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118612 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118631 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118636 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118640 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118644 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118650 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118655 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118659 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118664 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118668 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118672 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118677 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118681 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118686 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118689 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118696 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118701 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.118706 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 07:27:47.121815 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119825 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119836 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119843 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119849 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119856 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119862 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119866 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119871 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119875 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119879 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119883 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119887 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119892 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119896 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119900 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119904 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119908 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119912 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119916 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119920 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 07:27:47.122486 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119924 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119928 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119932 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119936 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119941 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119945 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119950 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119954 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119958 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119962 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119966 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119971 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119974 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119979 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119984 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119989 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119994 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.119998 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120003 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 07:27:47.123005 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120007 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120011 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120016 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120020 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120024 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120028 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120032 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120037 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120041 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120045 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120049 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120053 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120057 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120060 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120064 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120068 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120074 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120079 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120082 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120087 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 07:27:47.123477 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120094 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120100 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120105 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120109 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120114 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120119 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120123 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120131 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120137 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120142 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120147 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120152 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120161 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120165 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120169 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120173 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120177 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120182 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120186 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 07:27:47.124051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120190 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120194 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120199 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120203 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120207 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120212 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120216 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120220 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120225 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120229 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120233 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120237 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120241 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120246 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120250 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120254 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120258 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120262 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120266 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120271 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 07:27:47.124516 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120275 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120848 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120857 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120863 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120868 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120877 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120882 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120887 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120891 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120896 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120899 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120904 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120908 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120912 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120916 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120920 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120924 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120928 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120932 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120937 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 07:27:47.125018 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120942 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120947 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120951 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120955 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120959 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120963 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120967 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120971 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120976 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120980 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120987 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120993 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.120997 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121002 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121006 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121011 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121015 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121023 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121028 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 07:27:47.125524 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121032 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121036 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121041 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121045 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121049 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121053 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121057 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121061 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121065 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121070 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121074 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121079 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121083 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121088 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121092 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121096 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121100 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121104 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121109 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121113 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 07:27:47.126009 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121117 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121121 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121125 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121129 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121134 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121138 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121142 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121147 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121151 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121155 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121161 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121165 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121170 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121174 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121178 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121182 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121186 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121190 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121194 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121198 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 07:27:47.126536 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121202 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121206 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121212 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121217 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121223 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121228 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121233 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121238 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121243 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121248 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121252 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121256 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121260 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121264 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121271 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121275 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121279 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121283 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121287 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 07:27:47.127059 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121292 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.121296 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121400 2580 flags.go:64] FLAG: --address="0.0.0.0" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121448 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121459 2580 flags.go:64] FLAG: --anonymous-auth="true" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121466 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121473 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121479 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121485 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121492 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121498 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121502 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121508 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121513 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121519 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121524 2580 flags.go:64] FLAG: --cgroup-root="" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121528 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121533 2580 flags.go:64] FLAG: --client-ca-file="" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121538 2580 flags.go:64] FLAG: --cloud-provider="external" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121543 2580 flags.go:64] FLAG: --cluster-dns="[]" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121549 2580 flags.go:64] FLAG: --cluster-domain="" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121554 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121559 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121564 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 16 07:27:47.127515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121569 2580 flags.go:64] FLAG: --container-log-max-files="5" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121575 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121580 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121587 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121593 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121598 2580 flags.go:64] FLAG: --contention-profiling="false" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121602 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121607 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121612 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121634 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121641 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121649 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121654 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121658 2580 flags.go:64] FLAG: --enable-load-reader="false" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121663 2580 flags.go:64] FLAG: --enable-server="true" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121669 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121676 2580 flags.go:64] FLAG: --event-burst="100" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121681 2580 flags.go:64] FLAG: --event-qps="50" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121686 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121691 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121695 2580 flags.go:64] FLAG: --eviction-hard="" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121701 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121707 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121711 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121716 2580 flags.go:64] FLAG: --eviction-soft="" Sep 16 07:27:47.128133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121721 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121726 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121731 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121735 2580 flags.go:64] FLAG: --experimental-mounter-path="" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121740 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121745 2580 flags.go:64] FLAG: --fail-swap-on="true" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121750 2580 flags.go:64] FLAG: --feature-gates="" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121760 2580 flags.go:64] FLAG: --file-check-frequency="20s" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121764 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121770 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121776 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121782 2580 flags.go:64] FLAG: --healthz-port="10248" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121787 2580 flags.go:64] FLAG: --help="false" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121791 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121796 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121801 2580 flags.go:64] FLAG: --http-check-frequency="20s" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121806 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121811 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121819 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121824 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121828 2580 flags.go:64] FLAG: --image-service-endpoint="" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121833 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121837 2580 flags.go:64] FLAG: --kube-api-burst="100" Sep 16 07:27:47.128792 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121851 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121857 2580 flags.go:64] FLAG: --kube-api-qps="50" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121862 2580 flags.go:64] FLAG: --kube-reserved="" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121867 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121872 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121877 2580 flags.go:64] FLAG: --kubelet-cgroups="" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121881 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121886 2580 flags.go:64] FLAG: --lock-file="" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121890 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121895 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121900 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121921 2580 flags.go:64] FLAG: --log-json-split-stream="false" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121926 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121930 2580 flags.go:64] FLAG: --log-text-split-stream="false" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121935 2580 flags.go:64] FLAG: --logging-format="text" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121940 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121946 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121950 2580 flags.go:64] FLAG: --manifest-url="" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121955 2580 flags.go:64] FLAG: --manifest-url-header="" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121963 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121991 2580 flags.go:64] FLAG: --max-open-files="1000000" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.121999 2580 flags.go:64] FLAG: --max-pods="110" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122005 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122010 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122015 2580 flags.go:64] FLAG: --memory-manager-policy="None" Sep 16 07:27:47.129389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122021 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122026 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122031 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122036 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122046 2580 flags.go:64] FLAG: --node-status-max-images="50" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122050 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122055 2580 flags.go:64] FLAG: --oom-score-adj="-999" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122060 2580 flags.go:64] FLAG: --pod-cidr="" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122065 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122074 2580 flags.go:64] FLAG: --pod-manifest-path="" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122079 2580 flags.go:64] FLAG: --pod-max-pids="-1" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122084 2580 flags.go:64] FLAG: --pods-per-core="0" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122088 2580 flags.go:64] FLAG: --port="10250" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122093 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122097 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bf9cc2faf94ce979" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122103 2580 flags.go:64] FLAG: --qos-reserved="" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122108 2580 flags.go:64] FLAG: --read-only-port="10255" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122113 2580 flags.go:64] FLAG: --register-node="true" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122118 2580 flags.go:64] FLAG: --register-with-taints="" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122124 2580 flags.go:64] FLAG: --registry-burst="10" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122128 2580 flags.go:64] FLAG: --registry-qps="5" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122133 2580 flags.go:64] FLAG: --reserved-cpus="" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122138 2580 flags.go:64] FLAG: --reserved-memory="" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122144 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122150 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 16 07:27:47.130025 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122154 2580 flags.go:64] FLAG: --rotate-certificates="false" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122159 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122165 2580 flags.go:64] FLAG: --runonce="false" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122170 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122175 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122180 2580 flags.go:64] FLAG: --seccomp-default="false" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122184 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122189 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122194 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122200 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122205 2580 flags.go:64] FLAG: --storage-driver-password="root" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122210 2580 flags.go:64] FLAG: --storage-driver-secure="false" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122215 2580 flags.go:64] FLAG: --storage-driver-table="stats" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122220 2580 flags.go:64] FLAG: --storage-driver-user="root" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122224 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122229 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122235 2580 flags.go:64] FLAG: --system-cgroups="" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122240 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122284 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122291 2580 flags.go:64] FLAG: --tls-cert-file="" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122294 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122301 2580 flags.go:64] FLAG: --tls-min-version="" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122305 2580 flags.go:64] FLAG: --tls-private-key-file="" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122308 2580 flags.go:64] FLAG: --topology-manager-policy="none" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122311 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 16 07:27:47.130680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122315 2580 flags.go:64] FLAG: --topology-manager-scope="container" Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122319 2580 flags.go:64] FLAG: --v="2" Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122324 2580 flags.go:64] FLAG: --version="false" Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122328 2580 flags.go:64] FLAG: --vmodule="" Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122332 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.122335 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122458 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122463 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122466 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122469 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122472 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122474 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122477 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122480 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122482 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122485 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122487 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122492 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122495 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122498 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122502 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 07:27:47.131289 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122505 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122507 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122510 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122513 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122516 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122520 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122522 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122525 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122527 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122530 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122533 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122535 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122538 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122541 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122543 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122546 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122549 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122552 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122554 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122557 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 07:27:47.131821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122559 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122562 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122564 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122567 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122570 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122573 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122575 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122578 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122580 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122583 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122585 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122588 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122591 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122593 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122596 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122598 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122604 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122607 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122610 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122613 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 07:27:47.132313 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122615 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122632 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122635 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122638 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122640 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122643 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122646 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122649 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122651 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122654 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122657 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122659 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122662 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122665 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122668 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122670 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122673 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122676 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122679 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122681 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 07:27:47.132821 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122684 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122688 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122692 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122695 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122697 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122700 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122703 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122706 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122711 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122714 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122717 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122721 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122725 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122728 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122731 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122734 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122736 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122739 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122741 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 07:27:47.133319 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122744 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122746 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122749 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122752 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.122754 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.123400 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.130152 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.130166 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.130183 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.130189 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130245 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130249 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130253 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130256 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 07:27:47.133866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130258 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130261 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130264 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130267 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130270 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130273 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130276 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130279 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130281 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130284 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130288 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130291 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130293 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130296 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130299 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130301 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130304 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130307 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130309 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130312 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 07:27:47.134220 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130315 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130317 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130320 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130322 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130325 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130329 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130331 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130334 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130337 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130341 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130345 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130348 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130351 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130354 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130356 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130359 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130362 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130364 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130367 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 07:27:47.134720 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130370 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130372 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130375 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130377 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130381 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130384 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130386 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130389 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130391 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130403 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130407 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130410 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130413 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130415 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130418 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130420 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130423 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130426 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130429 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 07:27:47.135185 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130431 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130434 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130437 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130439 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130442 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130445 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130447 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130450 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130452 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130455 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130459 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130462 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130465 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130467 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130470 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130472 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130475 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130477 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130481 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130483 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 07:27:47.135654 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130486 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130488 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130491 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130493 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130496 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130498 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130501 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130503 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130507 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130510 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130513 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130516 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130519 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130522 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130525 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130528 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 07:27:47.136142 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130530 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.130535 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130656 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130661 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130664 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130667 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130669 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130672 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130676 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130680 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130683 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130686 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130689 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130692 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130695 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 07:27:47.136530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130698 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130701 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130704 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130706 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130709 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130711 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130714 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130717 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130719 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130722 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130724 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130727 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130729 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130732 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130735 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130737 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130740 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130742 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130745 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130747 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 07:27:47.136916 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130750 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130753 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130755 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130758 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130760 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130763 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130765 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130768 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130770 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130773 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130775 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130778 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130781 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130783 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130786 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130789 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130791 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130794 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130796 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130799 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 07:27:47.137400 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130801 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130804 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130806 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130809 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130811 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130815 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130817 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130820 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130822 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130825 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130828 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130830 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130833 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130836 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130838 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130841 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130844 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130846 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130848 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130851 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 07:27:47.137883 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130853 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130856 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130858 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130861 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130863 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130866 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130869 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130872 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130874 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130877 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130879 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130882 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130885 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130888 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130891 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130894 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130897 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130899 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130901 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 07:27:47.138358 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130904 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130906 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130910 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130915 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130918 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130921 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:47.130923 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.130942 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.131031 2580 server.go:962] "Client rotation is on, will bootstrap in background" Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.132859 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.133976 2580 server.go:1019] "Starting client certificate rotation" Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.134072 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.134110 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 16 07:27:47.138836 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.137191 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 16 07:27:47.142975 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.142955 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 16 07:27:47.152861 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.152846 2580 log.go:25] "Validated CRI v1 runtime API" Sep 16 07:27:47.155782 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.155768 2580 log.go:25] "Validated CRI v1 image API" Sep 16 07:27:47.156660 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.156635 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 16 07:27:47.159240 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.159223 2580 fs.go:135] Filesystem UUIDs: map[218e9330-ff4e-4b10-8db1-2fcdc5eacf2f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8e360585-a497-4d8c-b535-801b1e003fce:/dev/nvme0n1p3] Sep 16 07:27:47.159288 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.159240 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 16 07:27:47.164525 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.164421 2580 manager.go:216] Machine: {Timestamp:2026-09-16 07:27:47.163201771 +0000 UTC m=+0.290722872 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3130480 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208dc587819475f1af2c78e75cab41 SystemUUID:ec208dc5-8781-9475-f1af-2c78e75cab41 BootID:34c8ae3b-a5f6-44e2-b2b7-90a41a1c9885 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:dc:9e:de:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:dc:9e:de:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:6f:a2:bb:ea:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 16 07:27:47.164525 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.164519 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 16 07:27:47.164652 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.164613 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 16 07:27:47.167423 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.167398 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 16 07:27:47.167553 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.167425 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-65.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 16 07:27:47.168588 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.168578 2580 topology_manager.go:138] "Creating topology manager with none policy" Sep 16 07:27:47.168642 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.168591 2580 container_manager_linux.go:309] "Creating device plugin manager" Sep 16 07:27:47.168642 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.168602 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 16 07:27:47.169177 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.169167 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 16 07:27:47.170163 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.170153 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 16 07:27:47.173735 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.173724 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 16 07:27:47.173844 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.173835 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 16 07:27:47.173880 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.173871 2580 kubelet.go:486] "Attempting to sync node with API server" Sep 16 07:27:47.173909 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.173894 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 16 07:27:47.173909 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.173908 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 16 07:27:47.173958 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.173917 2580 kubelet.go:398] "Adding apiserver pod source" Sep 16 07:27:47.173958 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.173933 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 16 07:27:47.174317 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.174305 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 16 07:27:47.174359 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.174344 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 16 07:27:47.176655 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.176640 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 16 07:27:47.178999 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.178987 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 16 07:27:47.179037 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.179016 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 16 07:27:47.180633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180605 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180642 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180649 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180654 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180663 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180669 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180683 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180688 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180694 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180699 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180708 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 16 07:27:47.180712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.180713 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 16 07:27:47.182179 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.182167 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 16 07:27:47.182179 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.182177 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 16 07:27:47.183159 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.183140 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 16 07:27:47.185225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.185186 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-65.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 16 07:27:47.185392 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.185351 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 16 07:27:47.185661 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.185609 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-65.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 16 07:27:47.186697 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.186671 2580 server.go:1268] "Started kubelet" Sep 16 07:27:47.187102 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.187057 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 16 07:27:47.187721 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.187061 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 16 07:27:47.187721 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.187571 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 16 07:27:47.187963 ip-10-0-136-65 systemd[1]: Started Kubernetes Kubelet. Sep 16 07:27:47.189216 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.189189 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 16 07:27:47.191189 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.191149 2580 server.go:310] "Adding debug handlers to kubelet server" Sep 16 07:27:47.193908 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.193888 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 16 07:27:47.197718 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.196675 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-65.ec2.internal.18d5bc7be0b3e8ad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-65.ec2.internal,UID:ip-10-0-136-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-65.ec2.internal,},FirstTimestamp:2026-09-16 07:27:47.186526381 +0000 UTC m=+0.314047484,LastTimestamp:2026-09-16 07:27:47.186526381 +0000 UTC m=+0.314047484,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-65.ec2.internal,}" Sep 16 07:27:47.199077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.199059 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 16 07:27:47.199472 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.199454 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 16 07:27:47.199644 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.199610 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:47.200107 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200092 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 16 07:27:47.200107 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200095 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 16 07:27:47.200235 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200115 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 16 07:27:47.200235 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200197 2580 reconstruct.go:97] "Volume reconstruction finished" Sep 16 07:27:47.200235 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200205 2580 reconciler.go:29] "Reconciler: start to sync state" Sep 16 07:27:47.200365 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200299 2580 factory.go:55] Registering systemd factory Sep 16 07:27:47.200365 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200315 2580 factory.go:223] Registration of the systemd container factory successfully Sep 16 07:27:47.200525 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200513 2580 factory.go:153] Registering CRI-O factory Sep 16 07:27:47.200525 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200526 2580 factory.go:223] Registration of the crio container factory successfully Sep 16 07:27:47.200588 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200564 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 16 07:27:47.200588 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200577 2580 factory.go:103] Registering Raw factory Sep 16 07:27:47.200588 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200589 2580 manager.go:1196] Started watching for new ooms in manager Sep 16 07:27:47.200952 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.200942 2580 manager.go:349] Starting recovery of all containers Sep 16 07:27:47.207134 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.207107 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gmhz2" Sep 16 07:27:47.211146 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.211118 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 16 07:27:47.211244 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.211163 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-65.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 16 07:27:47.211917 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.211904 2580 manager.go:354] Recovery completed Sep 16 07:27:47.216192 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.216180 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 07:27:47.218319 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.218300 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gmhz2" Sep 16 07:27:47.218418 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.218346 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientMemory" Sep 16 07:27:47.218418 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.218377 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 07:27:47.218418 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.218401 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientPID" Sep 16 07:27:47.219069 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.219054 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 16 07:27:47.219069 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.219066 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 16 07:27:47.219069 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.219067 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 07:27:47.219206 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.219081 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 16 07:27:47.219781 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.219765 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientMemory" Sep 16 07:27:47.219866 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.219795 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 07:27:47.219866 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.219809 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientPID" Sep 16 07:27:47.229067 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.228953 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-65.ec2.internal.18d5bc7be2999c2f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-65.ec2.internal,UID:ip-10-0-136-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-65.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-65.ec2.internal,},FirstTimestamp:2026-09-16 07:27:47.218357295 +0000 UTC m=+0.345878399,LastTimestamp:2026-09-16 07:27:47.218357295 +0000 UTC m=+0.345878399,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-65.ec2.internal,}" Sep 16 07:27:47.230972 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.230909 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-65.ec2.internal.18d5bc7be29a04d4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-65.ec2.internal,UID:ip-10-0-136-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-65.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-65.ec2.internal,},FirstTimestamp:2026-09-16 07:27:47.218384084 +0000 UTC m=+0.345905187,LastTimestamp:2026-09-16 07:27:47.218384084 +0000 UTC m=+0.345905187,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-65.ec2.internal,}" Sep 16 07:27:47.240090 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.240017 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-65.ec2.internal.18d5bc7be29a5e7f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-65.ec2.internal,UID:ip-10-0-136-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-65.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-65.ec2.internal,},FirstTimestamp:2026-09-16 07:27:47.218407039 +0000 UTC m=+0.345928141,LastTimestamp:2026-09-16 07:27:47.218407039 +0000 UTC m=+0.345928141,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-65.ec2.internal,}" Sep 16 07:27:47.251396 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.251339 2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-65.ec2.internal.18d5bc7be2999c2f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-65.ec2.internal.18d5bc7be2999c2f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-65.ec2.internal,UID:ip-10-0-136-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-65.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-65.ec2.internal,},FirstTimestamp:2026-09-16 07:27:47.218357295 +0000 UTC m=+0.345878399,LastTimestamp:2026-09-16 07:27:47.219780168 +0000 UTC m=+0.347301276,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-65.ec2.internal,}" Sep 16 07:27:47.259333 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.259282 2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-65.ec2.internal.18d5bc7be29a04d4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-65.ec2.internal.18d5bc7be29a04d4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-65.ec2.internal,UID:ip-10-0-136-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-65.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-65.ec2.internal,},FirstTimestamp:2026-09-16 07:27:47.218384084 +0000 UTC m=+0.345905187,LastTimestamp:2026-09-16 07:27:47.219801976 +0000 UTC m=+0.347323079,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-65.ec2.internal,}" Sep 16 07:27:47.268188 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.268133 2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-65.ec2.internal.18d5bc7be29a5e7f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-65.ec2.internal.18d5bc7be29a5e7f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-65.ec2.internal,UID:ip-10-0-136-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-65.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-65.ec2.internal,},FirstTimestamp:2026-09-16 07:27:47.218407039 +0000 UTC m=+0.345928141,LastTimestamp:2026-09-16 07:27:47.219814948 +0000 UTC m=+0.347336051,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-65.ec2.internal,}" Sep 16 07:27:47.300289 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.300273 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:47.362292 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.362247 2580 policy_none.go:49] "None policy: Start" Sep 16 07:27:47.362292 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.362270 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 16 07:27:47.362292 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.362281 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 16 07:27:47.364771 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.364757 2580 policy_none.go:47] "Start" Sep 16 07:27:47.396246 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.396139 2580 manager.go:337] "Starting Device Plugin manager" Sep 16 07:27:47.396368 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.396272 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 16 07:27:47.396368 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.396285 2580 server.go:87] "Starting device plugin registration server" Sep 16 07:27:47.396493 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.396481 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 16 07:27:47.396545 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.396493 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 16 07:27:47.396642 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.396600 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 16 07:27:47.396717 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.396700 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 16 07:27:47.396717 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.396708 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 16 07:27:47.397180 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.397160 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 16 07:27:47.397252 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.397214 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:47.418209 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.418188 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-65.ec2.internal\" not found" node="ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.449646 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.449602 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 16 07:27:47.450714 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.450700 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 16 07:27:47.450790 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.450743 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 16 07:27:47.450790 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.450763 2580 kubelet.go:2448] "Starting kubelet main sync loop" Sep 16 07:27:47.450892 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.450799 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 16 07:27:47.454689 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.454674 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 16 07:27:47.496999 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.496981 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 07:27:47.500539 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.500516 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientMemory" Sep 16 07:27:47.500639 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.500544 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 07:27:47.500639 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.500556 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientPID" Sep 16 07:27:47.500639 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.500575 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.509753 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.509737 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.509842 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.509759 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-65.ec2.internal\": node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:47.543899 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.543882 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:47.550900 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.550882 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal"] Sep 16 07:27:47.550949 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.550936 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 07:27:47.552241 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.552227 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientMemory" Sep 16 07:27:47.552320 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.552249 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 07:27:47.552320 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.552263 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientPID" Sep 16 07:27:47.554455 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.554443 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 07:27:47.554560 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.554546 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.554601 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.554572 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 07:27:47.555166 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.555152 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientMemory" Sep 16 07:27:47.555244 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.555173 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 07:27:47.555244 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.555188 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientPID" Sep 16 07:27:47.555244 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.555214 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientMemory" Sep 16 07:27:47.555244 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.555237 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 07:27:47.555422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.555251 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientPID" Sep 16 07:27:47.557467 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.557451 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.557545 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.557479 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 07:27:47.558132 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.558115 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientMemory" Sep 16 07:27:47.558199 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.558149 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 07:27:47.558199 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.558165 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientPID" Sep 16 07:27:47.583146 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.583123 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-65.ec2.internal\" not found" node="ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.587348 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.587335 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-65.ec2.internal\" not found" node="ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.644096 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.644060 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:47.701373 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.701350 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6832768f5caf72088d8c5421116df2db-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal\" (UID: \"6832768f5caf72088d8c5421116df2db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.701445 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.701380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6832768f5caf72088d8c5421116df2db-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal\" (UID: \"6832768f5caf72088d8c5421116df2db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.701445 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.701400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/98bcfe4ba9a3e44508035277e4f353da-config\") pod \"kube-apiserver-proxy-ip-10-0-136-65.ec2.internal\" (UID: \"98bcfe4ba9a3e44508035277e4f353da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.744425 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.744407 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:47.802008 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.801986 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6832768f5caf72088d8c5421116df2db-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal\" (UID: \"6832768f5caf72088d8c5421116df2db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.802059 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.802015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6832768f5caf72088d8c5421116df2db-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal\" (UID: \"6832768f5caf72088d8c5421116df2db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.802059 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.802031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/98bcfe4ba9a3e44508035277e4f353da-config\") pod \"kube-apiserver-proxy-ip-10-0-136-65.ec2.internal\" (UID: \"98bcfe4ba9a3e44508035277e4f353da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.802130 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.802085 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/98bcfe4ba9a3e44508035277e4f353da-config\") pod \"kube-apiserver-proxy-ip-10-0-136-65.ec2.internal\" (UID: \"98bcfe4ba9a3e44508035277e4f353da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.802130 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.802085 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6832768f5caf72088d8c5421116df2db-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal\" (UID: \"6832768f5caf72088d8c5421116df2db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.802130 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.802097 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6832768f5caf72088d8c5421116df2db-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal\" (UID: \"6832768f5caf72088d8c5421116df2db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.845108 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.845090 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:47.886284 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.886269 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.889801 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:47.889788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" Sep 16 07:27:47.945737 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:47.945668 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:48.046195 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:48.046177 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:49.736812 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:49.736781 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 16 07:27:49.737312 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:49.736913 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 16 07:27:49.750026 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:49.750006 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:49.850873 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:49.850820 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:49.870973 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:49.870958 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 07:27:49.871781 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:49.871765 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientMemory" Sep 16 07:27:49.871863 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:49.871812 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 07:27:49.871863 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:49.871823 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeHasSufficientPID" Sep 16 07:27:49.901885 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:49.901860 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 16 07:27:49.951040 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:49.951023 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:49.968189 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:49.968161 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 07:22:48 +0000 UTC" deadline="2028-03-16 00:07:09.250593826 +0000 UTC" Sep 16 07:27:49.968189 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:49.968186 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13120h39m19.282410965s" Sep 16 07:27:50.051770 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:50.051745 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:50.122546 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:50.122514 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bcfe4ba9a3e44508035277e4f353da.slice/crio-5fa2b69728cf7854282f7040b92503629821d1e13e1ed27f8dbc76e3b3dab2fb WatchSource:0}: Error finding container 5fa2b69728cf7854282f7040b92503629821d1e13e1ed27f8dbc76e3b3dab2fb: Status 404 returned error can't find the container with id 5fa2b69728cf7854282f7040b92503629821d1e13e1ed27f8dbc76e3b3dab2fb Sep 16 07:27:50.122831 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:50.122811 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6832768f5caf72088d8c5421116df2db.slice/crio-ce767b3f1037f4bc11c55de98255835d78457321406e6005f95287e574359c30 WatchSource:0}: Error finding container ce767b3f1037f4bc11c55de98255835d78457321406e6005f95287e574359c30: Status 404 returned error can't find the container with id ce767b3f1037f4bc11c55de98255835d78457321406e6005f95287e574359c30 Sep 16 07:27:50.127147 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.127128 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 16 07:27:50.152614 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:50.152589 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-65.ec2.internal\" not found" Sep 16 07:27:50.213939 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.213921 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 16 07:27:50.303471 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.303443 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal" Sep 16 07:27:50.310547 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.310530 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 16 07:27:50.311307 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.311296 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" Sep 16 07:27:50.324400 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.324382 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 16 07:27:50.332478 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.332462 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 16 07:27:50.589198 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.589136 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 16 07:27:50.779222 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.779197 2580 apiserver.go:52] "Watching apiserver" Sep 16 07:27:50.785768 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.785743 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 16 07:27:50.786306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.786242 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-6qkmk","kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st","openshift-cluster-node-tuning-operator/tuned-j6lxc","openshift-image-registry/node-ca-962g2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal","openshift-multus/multus-5mpgc","openshift-multus/multus-additional-cni-plugins-mq8vr","kube-system/global-pull-secret-syncer-crdsq","openshift-dns/node-resolver-hjlx2","openshift-multus/network-metrics-daemon-99h7c","openshift-network-diagnostics/network-check-target-zcss9","openshift-network-operator/iptables-alerter-qdfwd","openshift-ovn-kubernetes/ovnkube-node-9rxmw"] Sep 16 07:27:50.788993 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.788970 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.791125 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.791107 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.794091 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.794054 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.794322 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.794299 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 16 07:27:50.794586 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.794567 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 16 07:27:50.794813 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.794781 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 16 07:27:50.794895 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.794837 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 16 07:27:50.795066 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.795050 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jwkx8\"" Sep 16 07:27:50.795143 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.795095 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 16 07:27:50.795198 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.795050 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 16 07:27:50.795309 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.795293 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 16 07:27:50.795398 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.795380 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2jwjn\"" Sep 16 07:27:50.795533 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.795518 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.796892 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.796865 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 16 07:27:50.797632 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.797598 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-54qvk\"" Sep 16 07:27:50.797723 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.797667 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.798202 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.798181 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 16 07:27:50.798621 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.798601 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 16 07:27:50.799812 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.799734 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 16 07:27:50.800093 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.800077 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 16 07:27:50.800312 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.800294 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-knq6g\"" Sep 16 07:27:50.800695 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.800676 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.802414 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.802261 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:50.802414 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:50.802327 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:27:50.804225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.803099 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 16 07:27:50.804225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.803146 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 16 07:27:50.804225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.803203 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 16 07:27:50.804225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.803453 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9qd8s\"" Sep 16 07:27:50.804225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.803596 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 16 07:27:50.804876 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.804482 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:27:50.807678 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.807647 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:50.808923 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.808899 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 16 07:27:50.809439 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.809415 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 16 07:27:50.809862 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.809843 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 16 07:27:50.810421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.810399 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:50.810863 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.810844 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ghxhs\"" Sep 16 07:27:50.811236 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.811217 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 16 07:27:50.811309 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.811295 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8kmcf\"" Sep 16 07:27:50.811490 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.811472 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 16 07:27:50.811707 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.811688 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 16 07:27:50.811967 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.811949 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 16 07:27:50.812488 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.812466 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rtrh9\"" Sep 16 07:27:50.813251 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.812951 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 16 07:27:50.813251 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.813022 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 16 07:27:50.814003 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.813706 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9kmfp\"" Sep 16 07:27:50.814711 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.814692 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnkj4\" (UniqueName: \"kubernetes.io/projected/4c658af4-a137-4b9e-9573-b66f4b836ba9-kube-api-access-xnkj4\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.814880 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.814861 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-run-netns\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-etc-kubernetes\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-node-log\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815544 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-run-openvswitch\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-registration-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-etc-selinux\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815686 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-tuned\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815713 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-slash\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815750 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99e8b563-af63-4f93-b619-8a7e4c07603d-env-overrides\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815810 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c658af4-a137-4b9e-9573-b66f4b836ba9-cni-binary-copy\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-var-lib-cni-multus\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.815891 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:50.815983 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-sys-fs\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816076 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-systemd\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-system-cni-dir\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816164 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vskhz\" (UniqueName: \"kubernetes.io/projected/67eb3291-276f-4393-b1f0-85f4eb5b1887-kube-api-access-vskhz\") pod \"iptables-alerter-qdfwd\" (UID: \"67eb3291-276f-4393-b1f0-85f4eb5b1887\") " pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.816741 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-modprobe-d\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2-host\") pod \"node-ca-962g2\" (UID: \"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2\") " pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-cni-netd\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b4e05ee-c901-4380-91d5-714df46eec55-agent-certs\") pod \"konnectivity-agent-6qkmk\" (UID: \"4b4e05ee-c901-4380-91d5-714df46eec55\") " pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8535a87b-98d2-487c-9ed7-e9c9d4468e71-tmp\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816463 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-systemd-units\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816547 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99e8b563-af63-4f93-b619-8a7e4c07603d-ovnkube-script-lib\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-cnibin\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816661 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-var-lib-cni-bin\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-hostroot\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816718 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-run-multus-certs\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816779 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-run-ovn\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99e8b563-af63-4f93-b619-8a7e4c07603d-ovnkube-config\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.816901 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-conf-dir\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67eb3291-276f-4393-b1f0-85f4eb5b1887-iptables-alerter-script\") pod \"iptables-alerter-qdfwd\" (UID: \"67eb3291-276f-4393-b1f0-85f4eb5b1887\") " pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.817633 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-socket-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-var-lib-kubelet\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817188 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2-serviceca\") pod \"node-ca-962g2\" (UID: \"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2\") " pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817224 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-var-lib-openvswitch\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-cni-bin\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cbfcdb7d-38d2-4284-b416-750df96a620a-kubelet-config\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817316 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-run-k8s-cni-cncf-io\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-etc-openvswitch\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99e8b563-af63-4f93-b619-8a7e4c07603d-ovn-node-metrics-cert\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817480 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cbfcdb7d-38d2-4284-b416-750df96a620a-dbus\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-run-netns\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-daemon-config\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-kubernetes\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817613 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-run-ovn-kubernetes\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.818364 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.817642 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-device-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.819028 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.818678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-sys\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.819028 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.818709 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-lib-modules\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.819028 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.818732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-log-socket\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.819028 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.818758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbkpn\" (UniqueName: \"kubernetes.io/projected/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-kube-api-access-kbkpn\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.819028 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.818827 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjdfb\" (UniqueName: \"kubernetes.io/projected/28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2-kube-api-access-kjdfb\") pod \"node-ca-962g2\" (UID: \"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2\") " pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.819028 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.818882 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-socket-dir-parent\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.819028 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.818910 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-sysctl-d\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.819028 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.818950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-sysctl-conf\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.818998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsqvl\" (UniqueName: \"kubernetes.io/projected/8535a87b-98d2-487c-9ed7-e9c9d4468e71-kube-api-access-qsqvl\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fxkj\" (UniqueName: \"kubernetes.io/projected/99e8b563-af63-4f93-b619-8a7e4c07603d-kube-api-access-2fxkj\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819126 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-cni-dir\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819145 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-var-lib-kubelet\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819171 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-sysconfig\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67eb3291-276f-4393-b1f0-85f4eb5b1887-host-slash\") pod \"iptables-alerter-qdfwd\" (UID: \"67eb3291-276f-4393-b1f0-85f4eb5b1887\") " pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819282 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-run\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819299 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-host\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819311 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-kubelet\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-run-systemd\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.819421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.819389 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b4e05ee-c901-4380-91d5-714df46eec55-konnectivity-ca\") pod \"konnectivity-agent-6qkmk\" (UID: \"4b4e05ee-c901-4380-91d5-714df46eec55\") " pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:27:50.820173 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.820151 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 16 07:27:50.820681 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.820583 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:50.820681 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:50.820647 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:27:50.822424 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.822125 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 16 07:27:50.822975 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.822935 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 16 07:27:50.831988 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.831969 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 16 07:27:50.897126 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.897094 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s4xwm" Sep 16 07:27:50.904561 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.904542 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 16 07:27:50.917736 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.917650 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s4xwm" Sep 16 07:27:50.919686 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-run-netns\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.919806 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-daemon-config\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.919806 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-kubernetes\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.919806 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-run-ovn-kubernetes\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.919806 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919782 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d34d815-6ad3-4107-9499-e004e45cb6ba-system-cni-dir\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:50.920013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-device-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.920013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-sys\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.920013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-lib-modules\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.920013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-log-socket\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.920013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbkpn\" (UniqueName: \"kubernetes.io/projected/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-kube-api-access-kbkpn\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.920013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919932 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjdfb\" (UniqueName: \"kubernetes.io/projected/28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2-kube-api-access-kjdfb\") pod \"node-ca-962g2\" (UID: \"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2\") " pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.920013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919954 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d34d815-6ad3-4107-9499-e004e45cb6ba-cnibin\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:50.920013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.919977 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d34d815-6ad3-4107-9499-e004e45cb6ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:50.920013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-socket-dir-parent\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920022 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-sysctl-d\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-sysctl-conf\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsqvl\" (UniqueName: \"kubernetes.io/projected/8535a87b-98d2-487c-9ed7-e9c9d4468e71-kube-api-access-qsqvl\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920101 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fxkj\" (UniqueName: \"kubernetes.io/projected/99e8b563-af63-4f93-b619-8a7e4c07603d-kube-api-access-2fxkj\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv28r\" (UniqueName: \"kubernetes.io/projected/92f584a7-6d57-4f6b-9f18-b49f43a95e4e-kube-api-access-vv28r\") pod \"node-resolver-hjlx2\" (UID: \"92f584a7-6d57-4f6b-9f18-b49f43a95e4e\") " pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-cni-dir\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-var-lib-kubelet\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-sysconfig\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67eb3291-276f-4393-b1f0-85f4eb5b1887-host-slash\") pod \"iptables-alerter-qdfwd\" (UID: \"67eb3291-276f-4393-b1f0-85f4eb5b1887\") " pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-run\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-host\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-kubelet\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-run-systemd\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.920431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b4e05ee-c901-4380-91d5-714df46eec55-konnectivity-ca\") pod \"konnectivity-agent-6qkmk\" (UID: \"4b4e05ee-c901-4380-91d5-714df46eec55\") " pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920448 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4ld7\" (UniqueName: \"kubernetes.io/projected/6d34d815-6ad3-4107-9499-e004e45cb6ba-kube-api-access-h4ld7\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnkj4\" (UniqueName: \"kubernetes.io/projected/4c658af4-a137-4b9e-9573-b66f4b836ba9-kube-api-access-xnkj4\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920504 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-run-netns\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-etc-kubernetes\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-node-log\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-run-openvswitch\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-registration-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-etc-selinux\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-tuned\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920734 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-slash\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99e8b563-af63-4f93-b619-8a7e4c07603d-env-overrides\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920799 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c658af4-a137-4b9e-9573-b66f4b836ba9-cni-binary-copy\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920823 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67eb3291-276f-4393-b1f0-85f4eb5b1887-host-slash\") pod \"iptables-alerter-qdfwd\" (UID: \"67eb3291-276f-4393-b1f0-85f4eb5b1887\") " pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920872 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-run-netns\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920887 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-run\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920878 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-var-lib-cni-multus\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920954 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-host\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.921249 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920998 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-kubelet\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921038 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-run-systemd\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-cni-dir\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921180 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-var-lib-kubelet\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921230 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921281 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-sysconfig\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.920824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-var-lib-cni-multus\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-sys-fs\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-systemd\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921408 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-daemon-config\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/92f584a7-6d57-4f6b-9f18-b49f43a95e4e-hosts-file\") pod \"node-resolver-hjlx2\" (UID: \"92f584a7-6d57-4f6b-9f18-b49f43a95e4e\") " pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921477 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-socket-dir-parent\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6d34d815-6ad3-4107-9499-e004e45cb6ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921458 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-sys-fs\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-system-cni-dir\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-lib-modules\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921578 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-etc-kubernetes\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921578 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b4e05ee-c901-4380-91d5-714df46eec55-konnectivity-ca\") pod \"konnectivity-agent-6qkmk\" (UID: \"4b4e05ee-c901-4380-91d5-714df46eec55\") " pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:27:50.922053 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921607 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-sysctl-d\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921620 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-run-openvswitch\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921671 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-log-socket\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921677 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-system-cni-dir\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921713 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-sysctl-conf\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921719 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-slash\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vskhz\" (UniqueName: \"kubernetes.io/projected/67eb3291-276f-4393-b1f0-85f4eb5b1887-kube-api-access-vskhz\") pod \"iptables-alerter-qdfwd\" (UID: \"67eb3291-276f-4393-b1f0-85f4eb5b1887\") " pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921773 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-etc-selinux\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-modprobe-d\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921843 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-registration-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921850 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-sys\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921610 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-run-ovn-kubernetes\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921871 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-systemd\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2-host\") pod \"node-ca-962g2\" (UID: \"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2\") " pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921888 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-run-netns\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921899 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-cni-netd\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921902 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-node-log\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.922943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921932 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b4e05ee-c901-4380-91d5-714df46eec55-agent-certs\") pod \"konnectivity-agent-6qkmk\" (UID: \"4b4e05ee-c901-4380-91d5-714df46eec55\") " pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6d34d815-6ad3-4107-9499-e004e45cb6ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxmxs\" (UniqueName: \"kubernetes.io/projected/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-kube-api-access-qxmxs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8535a87b-98d2-487c-9ed7-e9c9d4468e71-tmp\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-systemd-units\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922036 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-modprobe-d\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922049 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99e8b563-af63-4f93-b619-8a7e4c07603d-ovnkube-script-lib\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.921982 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-device-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-cnibin\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922124 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-var-lib-cni-bin\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-hostroot\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-run-multus-certs\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-run-ovn\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99e8b563-af63-4f93-b619-8a7e4c07603d-ovnkube-config\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922215 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:50.923729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-conf-dir\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67eb3291-276f-4393-b1f0-85f4eb5b1887-iptables-alerter-script\") pod \"iptables-alerter-qdfwd\" (UID: \"67eb3291-276f-4393-b1f0-85f4eb5b1887\") " pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922272 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2-host\") pod \"node-ca-962g2\" (UID: \"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2\") " pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922305 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-run-ovn\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922303 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-multus-conf-dir\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-socket-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922343 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-var-lib-cni-bin\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-var-lib-kubelet\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922374 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-systemd-units\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922355 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922395 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2-serviceca\") pod \"node-ca-962g2\" (UID: \"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2\") " pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922400 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-hostroot\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922403 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c658af4-a137-4b9e-9573-b66f4b836ba9-cni-binary-copy\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-var-lib-openvswitch\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922517 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-socket-dir\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922819 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99e8b563-af63-4f93-b619-8a7e4c07603d-env-overrides\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922841 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99e8b563-af63-4f93-b619-8a7e4c07603d-ovnkube-config\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922881 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-kubernetes\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.924530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922891 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-cnibin\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-cni-bin\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922940 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-cni-bin\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.922972 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923008 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-host-cni-netd\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923022 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99e8b563-af63-4f93-b619-8a7e4c07603d-ovnkube-script-lib\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923049 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-var-lib-openvswitch\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cbfcdb7d-38d2-4284-b416-750df96a620a-kubelet-config\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923088 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8535a87b-98d2-487c-9ed7-e9c9d4468e71-var-lib-kubelet\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-run-k8s-cni-cncf-io\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923138 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cbfcdb7d-38d2-4284-b416-750df96a620a-kubelet-config\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923179 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-etc-openvswitch\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99e8b563-af63-4f93-b619-8a7e4c07603d-ovn-node-metrics-cert\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923232 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2-serviceca\") pod \"node-ca-962g2\" (UID: \"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2\") " pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cbfcdb7d-38d2-4284-b416-750df96a620a-dbus\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923234 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-run-multus-certs\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923267 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4c658af4-a137-4b9e-9573-b66f4b836ba9-host-run-k8s-cni-cncf-io\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:50.925422 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923288 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cbfcdb7d-38d2-4284-b416-750df96a620a-dbus\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:50.926225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923304 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e8b563-af63-4f93-b619-8a7e4c07603d-etc-openvswitch\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.926225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f584a7-6d57-4f6b-9f18-b49f43a95e4e-tmp-dir\") pod \"node-resolver-hjlx2\" (UID: \"92f584a7-6d57-4f6b-9f18-b49f43a95e4e\") " pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:50.926225 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:50.923352 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:50.926225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923388 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67eb3291-276f-4393-b1f0-85f4eb5b1887-iptables-alerter-script\") pod \"iptables-alerter-qdfwd\" (UID: \"67eb3291-276f-4393-b1f0-85f4eb5b1887\") " pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.926225 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.923401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d34d815-6ad3-4107-9499-e004e45cb6ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:50.926225 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:50.923482 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret podName:cbfcdb7d-38d2-4284-b416-750df96a620a nodeName:}" failed. No retries permitted until 2026-09-16 07:27:51.423448061 +0000 UTC m=+2.947801824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret") pod "global-pull-secret-syncer-crdsq" (UID: "cbfcdb7d-38d2-4284-b416-750df96a620a") : object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:50.928315 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.927643 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b4e05ee-c901-4380-91d5-714df46eec55-agent-certs\") pod \"konnectivity-agent-6qkmk\" (UID: \"4b4e05ee-c901-4380-91d5-714df46eec55\") " pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:27:50.928315 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.928270 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8535a87b-98d2-487c-9ed7-e9c9d4468e71-etc-tuned\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.929730 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.929711 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99e8b563-af63-4f93-b619-8a7e4c07603d-ovn-node-metrics-cert\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.930116 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.930094 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8535a87b-98d2-487c-9ed7-e9c9d4468e71-tmp\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.948838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.942124 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fxkj\" (UniqueName: \"kubernetes.io/projected/99e8b563-af63-4f93-b619-8a7e4c07603d-kube-api-access-2fxkj\") pod \"ovnkube-node-9rxmw\" (UID: \"99e8b563-af63-4f93-b619-8a7e4c07603d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:50.948838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.942195 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vskhz\" (UniqueName: \"kubernetes.io/projected/67eb3291-276f-4393-b1f0-85f4eb5b1887-kube-api-access-vskhz\") pod \"iptables-alerter-qdfwd\" (UID: \"67eb3291-276f-4393-b1f0-85f4eb5b1887\") " pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:50.948838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.942759 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsqvl\" (UniqueName: \"kubernetes.io/projected/8535a87b-98d2-487c-9ed7-e9c9d4468e71-kube-api-access-qsqvl\") pod \"tuned-j6lxc\" (UID: \"8535a87b-98d2-487c-9ed7-e9c9d4468e71\") " pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:50.948838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.946587 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjdfb\" (UniqueName: \"kubernetes.io/projected/28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2-kube-api-access-kjdfb\") pod \"node-ca-962g2\" (UID: \"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2\") " pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:50.948838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.946890 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbkpn\" (UniqueName: \"kubernetes.io/projected/06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b-kube-api-access-kbkpn\") pod \"aws-ebs-csi-driver-node-b24st\" (UID: \"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:50.948838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:50.948771 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnkj4\" (UniqueName: \"kubernetes.io/projected/4c658af4-a137-4b9e-9573-b66f4b836ba9-kube-api-access-xnkj4\") pod \"multus-5mpgc\" (UID: \"4c658af4-a137-4b9e-9573-b66f4b836ba9\") " pod="openshift-multus/multus-5mpgc" Sep 16 07:27:51.024673 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4ld7\" (UniqueName: \"kubernetes.io/projected/6d34d815-6ad3-4107-9499-e004e45cb6ba-kube-api-access-h4ld7\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.024814 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024692 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/92f584a7-6d57-4f6b-9f18-b49f43a95e4e-hosts-file\") pod \"node-resolver-hjlx2\" (UID: \"92f584a7-6d57-4f6b-9f18-b49f43a95e4e\") " pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:51.024814 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6d34d815-6ad3-4107-9499-e004e45cb6ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.024814 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6d34d815-6ad3-4107-9499-e004e45cb6ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.024814 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024771 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxmxs\" (UniqueName: \"kubernetes.io/projected/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-kube-api-access-qxmxs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:51.024814 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:51.025077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f584a7-6d57-4f6b-9f18-b49f43a95e4e-tmp-dir\") pod \"node-resolver-hjlx2\" (UID: \"92f584a7-6d57-4f6b-9f18-b49f43a95e4e\") " pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:51.025077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d34d815-6ad3-4107-9499-e004e45cb6ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.025077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d34d815-6ad3-4107-9499-e004e45cb6ba-system-cni-dir\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.025077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d34d815-6ad3-4107-9499-e004e45cb6ba-cnibin\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.025077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.024981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d34d815-6ad3-4107-9499-e004e45cb6ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.025077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.025007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv28r\" (UniqueName: \"kubernetes.io/projected/92f584a7-6d57-4f6b-9f18-b49f43a95e4e-kube-api-access-vv28r\") pod \"node-resolver-hjlx2\" (UID: \"92f584a7-6d57-4f6b-9f18-b49f43a95e4e\") " pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:51.025077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.025035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:51.025383 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.025223 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/92f584a7-6d57-4f6b-9f18-b49f43a95e4e-hosts-file\") pod \"node-resolver-hjlx2\" (UID: \"92f584a7-6d57-4f6b-9f18-b49f43a95e4e\") " pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:51.025383 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.025345 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f584a7-6d57-4f6b-9f18-b49f43a95e4e-tmp-dir\") pod \"node-resolver-hjlx2\" (UID: \"92f584a7-6d57-4f6b-9f18-b49f43a95e4e\") " pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:51.025738 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.025624 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:51.025738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.025695 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d34d815-6ad3-4107-9499-e004e45cb6ba-cnibin\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.025738 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.025713 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:27:51.525687556 +0000 UTC m=+3.050041316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:51.025924 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.025862 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6d34d815-6ad3-4107-9499-e004e45cb6ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.025924 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.025890 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d34d815-6ad3-4107-9499-e004e45cb6ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.026021 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.025932 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d34d815-6ad3-4107-9499-e004e45cb6ba-system-cni-dir\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.026021 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.025962 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6d34d815-6ad3-4107-9499-e004e45cb6ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.026397 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.026364 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d34d815-6ad3-4107-9499-e004e45cb6ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.043666 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.043623 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4ld7\" (UniqueName: \"kubernetes.io/projected/6d34d815-6ad3-4107-9499-e004e45cb6ba-kube-api-access-h4ld7\") pod \"multus-additional-cni-plugins-mq8vr\" (UID: \"6d34d815-6ad3-4107-9499-e004e45cb6ba\") " pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.044390 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.044365 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv28r\" (UniqueName: \"kubernetes.io/projected/92f584a7-6d57-4f6b-9f18-b49f43a95e4e-kube-api-access-vv28r\") pod \"node-resolver-hjlx2\" (UID: \"92f584a7-6d57-4f6b-9f18-b49f43a95e4e\") " pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:51.044732 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.044711 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 07:27:51.044834 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.044739 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 07:27:51.044834 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.044752 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vpln4 for pod openshift-network-diagnostics/network-check-target-zcss9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:51.044834 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.044832 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4 podName:d68eaab1-3586-4f06-8c86-d66a9908733a nodeName:}" failed. No retries permitted until 2026-09-16 07:27:51.544816619 +0000 UTC m=+3.069170407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vpln4" (UniqueName: "kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4") pod "network-check-target-zcss9" (UID: "d68eaab1-3586-4f06-8c86-d66a9908733a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:51.047640 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.047591 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxmxs\" (UniqueName: \"kubernetes.io/projected/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-kube-api-access-qxmxs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:51.058870 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.058818 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal" event={"ID":"98bcfe4ba9a3e44508035277e4f353da","Type":"ContainerStarted","Data":"5fa2b69728cf7854282f7040b92503629821d1e13e1ed27f8dbc76e3b3dab2fb"} Sep 16 07:27:51.060300 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.060277 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" event={"ID":"6832768f5caf72088d8c5421116df2db","Type":"ContainerStarted","Data":"ce767b3f1037f4bc11c55de98255835d78457321406e6005f95287e574359c30"} Sep 16 07:27:51.108403 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.108342 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5mpgc" Sep 16 07:27:51.119672 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.119651 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qdfwd" Sep 16 07:27:51.130190 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.130166 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" Sep 16 07:27:51.136800 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.136773 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" Sep 16 07:27:51.143389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.143371 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-962g2" Sep 16 07:27:51.151974 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.151958 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:27:51.159347 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.159326 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:27:51.168962 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.168944 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" Sep 16 07:27:51.174506 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.174485 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hjlx2" Sep 16 07:27:51.428418 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.428337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:51.428545 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.428487 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:51.428602 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.428546 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret podName:cbfcdb7d-38d2-4284-b416-750df96a620a nodeName:}" failed. No retries permitted until 2026-09-16 07:27:52.428530919 +0000 UTC m=+3.952884676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret") pod "global-pull-secret-syncer-crdsq" (UID: "cbfcdb7d-38d2-4284-b416-750df96a620a") : object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:51.529260 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.529231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:51.529425 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.529350 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:51.529425 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.529411 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:27:52.529394074 +0000 UTC m=+4.053747844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:51.629870 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.629844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:51.630048 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.630027 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 07:27:51.630101 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.630056 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 07:27:51.630101 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.630070 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vpln4 for pod openshift-network-diagnostics/network-check-target-zcss9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:51.630199 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:51.630127 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4 podName:d68eaab1-3586-4f06-8c86-d66a9908733a nodeName:}" failed. No retries permitted until 2026-09-16 07:27:52.630110941 +0000 UTC m=+4.154464720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpln4" (UniqueName: "kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4") pod "network-check-target-zcss9" (UID: "d68eaab1-3586-4f06-8c86-d66a9908733a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:51.921561 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.921468 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 07:22:50 +0000 UTC" deadline="2028-03-09 13:27:17.827128032 +0000 UTC" Sep 16 07:27:51.921561 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:51.921506 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12965h59m25.905626785s" Sep 16 07:27:52.054667 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:52.054637 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:52.054884 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:52.054645 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:52.054884 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.054807 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:27:52.054884 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.054847 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:27:52.074176 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:52.074044 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a8c6e2_ffc5_4951_9c7e_5d35d2d34dd2.slice/crio-c4e297383f414284b6e329d887f791dab5ff581fe691a90a8cbd8426ce9378f9 WatchSource:0}: Error finding container c4e297383f414284b6e329d887f791dab5ff581fe691a90a8cbd8426ce9378f9: Status 404 returned error can't find the container with id c4e297383f414284b6e329d887f791dab5ff581fe691a90a8cbd8426ce9378f9 Sep 16 07:27:52.074470 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:52.074442 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d34d815_6ad3_4107_9499_e004e45cb6ba.slice/crio-71e35c1d3031678e3004ec271238535274b17b8e08336a180011e5c691ef5d69 WatchSource:0}: Error finding container 71e35c1d3031678e3004ec271238535274b17b8e08336a180011e5c691ef5d69: Status 404 returned error can't find the container with id 71e35c1d3031678e3004ec271238535274b17b8e08336a180011e5c691ef5d69 Sep 16 07:27:52.080515 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:52.078733 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c658af4_a137_4b9e_9573_b66f4b836ba9.slice/crio-9dd17a14225c1702e194f043b36d656b31c1dac3a276911cdf5b18ad5fe84352 WatchSource:0}: Error finding container 9dd17a14225c1702e194f043b36d656b31c1dac3a276911cdf5b18ad5fe84352: Status 404 returned error can't find the container with id 9dd17a14225c1702e194f043b36d656b31c1dac3a276911cdf5b18ad5fe84352 Sep 16 07:27:52.082345 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:52.082324 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4e05ee_c901_4380_91d5_714df46eec55.slice/crio-216f2c3d9292476a1cf61f579d50f19571a46bbac9796fa012f05ebb940c37b4 WatchSource:0}: Error finding container 216f2c3d9292476a1cf61f579d50f19571a46bbac9796fa012f05ebb940c37b4: Status 404 returned error can't find the container with id 216f2c3d9292476a1cf61f579d50f19571a46bbac9796fa012f05ebb940c37b4 Sep 16 07:27:52.083866 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:52.083760 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92f584a7_6d57_4f6b_9f18_b49f43a95e4e.slice/crio-39bbbb10efc4f4dccfcaaf94ec402029e91719d81829e474b27379415b092fc2 WatchSource:0}: Error finding container 39bbbb10efc4f4dccfcaaf94ec402029e91719d81829e474b27379415b092fc2: Status 404 returned error can't find the container with id 39bbbb10efc4f4dccfcaaf94ec402029e91719d81829e474b27379415b092fc2 Sep 16 07:27:52.084529 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:27:52.084496 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06cf6a1f_84a9_4967_9ae0_f0dd8d05ec0b.slice/crio-c1ebcce5450095424c62fbc8855da08d0fdc92d9c0fa8e5f3ae8ba6e682b7ff3 WatchSource:0}: Error finding container c1ebcce5450095424c62fbc8855da08d0fdc92d9c0fa8e5f3ae8ba6e682b7ff3: Status 404 returned error can't find the container with id c1ebcce5450095424c62fbc8855da08d0fdc92d9c0fa8e5f3ae8ba6e682b7ff3 Sep 16 07:27:52.435885 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:52.435648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:52.436092 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.436035 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:52.436205 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.436108 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret podName:cbfcdb7d-38d2-4284-b416-750df96a620a nodeName:}" failed. No retries permitted until 2026-09-16 07:27:54.436087066 +0000 UTC m=+5.960440824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret") pod "global-pull-secret-syncer-crdsq" (UID: "cbfcdb7d-38d2-4284-b416-750df96a620a") : object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:52.537878 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:52.537212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:52.537878 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.537352 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:52.537878 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.537472 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:27:54.537451526 +0000 UTC m=+6.061805297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:52.638525 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:52.637890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:52.638525 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.638118 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 07:27:52.638525 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.638142 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 07:27:52.638525 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.638154 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vpln4 for pod openshift-network-diagnostics/network-check-target-zcss9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:52.638525 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:52.638207 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4 podName:d68eaab1-3586-4f06-8c86-d66a9908733a nodeName:}" failed. No retries permitted until 2026-09-16 07:27:54.638189818 +0000 UTC m=+6.162543595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpln4" (UniqueName: "kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4") pod "network-check-target-zcss9" (UID: "d68eaab1-3586-4f06-8c86-d66a9908733a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:52.922711 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:52.922604 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 07:22:50 +0000 UTC" deadline="2028-06-02 22:15:12.64994161 +0000 UTC" Sep 16 07:27:52.922711 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:52.922662 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15014h47m19.727283246s" Sep 16 07:27:53.056388 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.055906 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:53.056388 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:53.056025 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:27:53.071824 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.071752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6qkmk" event={"ID":"4b4e05ee-c901-4380-91d5-714df46eec55","Type":"ContainerStarted","Data":"216f2c3d9292476a1cf61f579d50f19571a46bbac9796fa012f05ebb940c37b4"} Sep 16 07:27:53.075252 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.075223 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5mpgc" event={"ID":"4c658af4-a137-4b9e-9573-b66f4b836ba9","Type":"ContainerStarted","Data":"9dd17a14225c1702e194f043b36d656b31c1dac3a276911cdf5b18ad5fe84352"} Sep 16 07:27:53.078681 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.078652 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" event={"ID":"6d34d815-6ad3-4107-9499-e004e45cb6ba","Type":"ContainerStarted","Data":"71e35c1d3031678e3004ec271238535274b17b8e08336a180011e5c691ef5d69"} Sep 16 07:27:53.085735 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.085708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-962g2" event={"ID":"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2","Type":"ContainerStarted","Data":"c4e297383f414284b6e329d887f791dab5ff581fe691a90a8cbd8426ce9378f9"} Sep 16 07:27:53.095547 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.095519 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal" event={"ID":"98bcfe4ba9a3e44508035277e4f353da","Type":"ContainerStarted","Data":"e526d040316ddd12a2f860befe73ce769824dd8f6090b6bd482517c91505ad1e"} Sep 16 07:27:53.101401 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.101266 2580 generic.go:356] "Generic (PLEG): container finished" podID="6832768f5caf72088d8c5421116df2db" containerID="4673561b71259c8cd211995caf1bd29b9aacaabded7f5ba7b63614fb2b51b21a" exitCode=0 Sep 16 07:27:53.101401 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.101347 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" event={"ID":"6832768f5caf72088d8c5421116df2db","Type":"ContainerDied","Data":"4673561b71259c8cd211995caf1bd29b9aacaabded7f5ba7b63614fb2b51b21a"} Sep 16 07:27:53.105178 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.105147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" event={"ID":"99e8b563-af63-4f93-b619-8a7e4c07603d","Type":"ContainerStarted","Data":"5a122d94820c7df9b948e124f3ae072edef13e6015aad3790598456e9a7e0832"} Sep 16 07:27:53.108970 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.108944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" event={"ID":"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b","Type":"ContainerStarted","Data":"c1ebcce5450095424c62fbc8855da08d0fdc92d9c0fa8e5f3ae8ba6e682b7ff3"} Sep 16 07:27:53.109274 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.109224 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-65.ec2.internal" podStartSLOduration=3.109210307 podStartE2EDuration="3.109210307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:27:53.108659102 +0000 UTC m=+4.633012884" watchObservedRunningTime="2026-09-16 07:27:53.109210307 +0000 UTC m=+4.633564087" Sep 16 07:27:53.113755 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.113729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qdfwd" event={"ID":"67eb3291-276f-4393-b1f0-85f4eb5b1887","Type":"ContainerStarted","Data":"54274b094ac34cdcd6bccd293df4a12315c1f939c77f5d26b059a3b3b7825ec1"} Sep 16 07:27:53.138994 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.137326 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hjlx2" event={"ID":"92f584a7-6d57-4f6b-9f18-b49f43a95e4e","Type":"ContainerStarted","Data":"39bbbb10efc4f4dccfcaaf94ec402029e91719d81829e474b27379415b092fc2"} Sep 16 07:27:53.139636 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:53.139611 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" event={"ID":"8535a87b-98d2-487c-9ed7-e9c9d4468e71","Type":"ContainerStarted","Data":"91a63f1390cb1d52f0b180ce977feb5426b11001ba9e2e7259dce07d69cce10c"} Sep 16 07:27:54.054779 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:54.054742 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:54.055267 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.054876 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:27:54.055267 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:54.054954 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:54.055267 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.055085 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:27:54.169872 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:54.169840 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" event={"ID":"6832768f5caf72088d8c5421116df2db","Type":"ContainerStarted","Data":"9f7ae5e986d3f20fb73a045118238d9adacc8ec3723a4b1ba5562c3e52ac8a05"} Sep 16 07:27:54.454696 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:54.454095 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:54.454696 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.454264 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:54.454696 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.454324 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret podName:cbfcdb7d-38d2-4284-b416-750df96a620a nodeName:}" failed. No retries permitted until 2026-09-16 07:27:58.454305911 +0000 UTC m=+9.978659681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret") pod "global-pull-secret-syncer-crdsq" (UID: "cbfcdb7d-38d2-4284-b416-750df96a620a") : object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:54.555557 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:54.555525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:54.555727 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.555683 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:54.555811 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.555746 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:27:58.555726234 +0000 UTC m=+10.080079996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:54.655959 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:54.655926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:54.656116 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.656066 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 07:27:54.656116 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.656083 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 07:27:54.656116 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.656095 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vpln4 for pod openshift-network-diagnostics/network-check-target-zcss9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:54.656280 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:54.656150 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4 podName:d68eaab1-3586-4f06-8c86-d66a9908733a nodeName:}" failed. No retries permitted until 2026-09-16 07:27:58.656132226 +0000 UTC m=+10.180485997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpln4" (UniqueName: "kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4") pod "network-check-target-zcss9" (UID: "d68eaab1-3586-4f06-8c86-d66a9908733a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:55.055931 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:55.055878 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:55.056541 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:55.056011 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:27:56.054953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:56.054921 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:56.055155 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:56.054969 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:56.055155 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:56.055050 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:27:56.055273 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:56.055185 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:27:57.055450 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:57.055386 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:57.055917 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:57.055524 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:27:58.054235 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:58.054150 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:58.054401 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:58.054150 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:58.054462 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.054395 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:27:58.054462 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.054300 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:27:58.484906 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:58.484824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:58.485354 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.484975 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:58.485354 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.485049 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret podName:cbfcdb7d-38d2-4284-b416-750df96a620a nodeName:}" failed. No retries permitted until 2026-09-16 07:28:06.48502531 +0000 UTC m=+18.009379069 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret") pod "global-pull-secret-syncer-crdsq" (UID: "cbfcdb7d-38d2-4284-b416-750df96a620a") : object "kube-system"/"original-pull-secret" not registered Sep 16 07:27:58.585906 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:58.585824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:27:58.586063 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.585971 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:58.586063 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.586035 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:28:06.58601618 +0000 UTC m=+18.110369955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:27:58.686844 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:58.686734 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:27:58.687002 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.686862 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 07:27:58.687002 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.686883 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 07:27:58.687002 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.686895 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vpln4 for pod openshift-network-diagnostics/network-check-target-zcss9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:58.687002 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:58.686948 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4 podName:d68eaab1-3586-4f06-8c86-d66a9908733a nodeName:}" failed. No retries permitted until 2026-09-16 07:28:06.686933074 +0000 UTC m=+18.211286845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpln4" (UniqueName: "kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4") pod "network-check-target-zcss9" (UID: "d68eaab1-3586-4f06-8c86-d66a9908733a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:27:59.055642 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:27:59.055610 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:27:59.055838 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:27:59.055720 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:00.055045 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:00.055010 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:00.055524 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:00.055133 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:00.055585 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:00.055524 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:00.055655 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:00.055629 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:01.054742 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:01.054704 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:01.054946 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:01.054857 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:02.054492 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:02.054461 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:02.055078 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:02.054462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:02.055078 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:02.054559 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:02.055078 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:02.054638 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:03.054935 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:03.054899 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:03.055334 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:03.055042 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:04.054427 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:04.054397 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:04.054606 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:04.054437 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:04.054606 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:04.054539 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:04.054763 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:04.054716 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:05.054606 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:05.054578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:05.055071 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:05.054696 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:06.054860 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:06.054826 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:06.055374 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.054934 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:06.055374 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:06.054999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:06.055374 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.055100 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:06.546212 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:06.546129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:06.546373 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.546280 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 07:28:06.546373 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.546357 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret podName:cbfcdb7d-38d2-4284-b416-750df96a620a nodeName:}" failed. No retries permitted until 2026-09-16 07:28:22.54633784 +0000 UTC m=+34.070691609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret") pod "global-pull-secret-syncer-crdsq" (UID: "cbfcdb7d-38d2-4284-b416-750df96a620a") : object "kube-system"/"original-pull-secret" not registered Sep 16 07:28:06.646623 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:06.646587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:06.646810 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.646713 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:28:06.646810 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.646775 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:28:22.646761364 +0000 UTC m=+34.171115121 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:28:06.747767 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:06.747726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:06.747988 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.747947 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 07:28:06.747988 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.747989 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 07:28:06.748161 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.748003 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vpln4 for pod openshift-network-diagnostics/network-check-target-zcss9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:28:06.748161 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:06.748065 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4 podName:d68eaab1-3586-4f06-8c86-d66a9908733a nodeName:}" failed. No retries permitted until 2026-09-16 07:28:22.748046773 +0000 UTC m=+34.272400551 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpln4" (UniqueName: "kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4") pod "network-check-target-zcss9" (UID: "d68eaab1-3586-4f06-8c86-d66a9908733a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:28:07.054579 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:07.054548 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:07.054736 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:07.054681 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:08.054677 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:08.054642 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:08.055076 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:08.054641 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:08.055076 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:08.054773 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:08.055076 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:08.054879 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:09.055256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:09.055216 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:09.055656 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:09.055350 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:10.054533 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.054196 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:10.054617 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:10.054592 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:10.054683 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.054204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:10.054776 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:10.054758 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:10.205044 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.205012 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" event={"ID":"99e8b563-af63-4f93-b619-8a7e4c07603d","Type":"ContainerStarted","Data":"e24457701c2369cb0a3b115fe1d08981034067edbee2e348ec1aa189274265fa"} Sep 16 07:28:10.205551 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.205111 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" event={"ID":"99e8b563-af63-4f93-b619-8a7e4c07603d","Type":"ContainerStarted","Data":"c291a288d1168ee1b9e5d55f3dcdf6d6cf629b4df088d647b6331b363417287e"} Sep 16 07:28:10.205551 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.205147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" event={"ID":"99e8b563-af63-4f93-b619-8a7e4c07603d","Type":"ContainerStarted","Data":"fedda3fb688e30bcdca25a00d3a9b4f8679478bd77ee100e9063c47647f5c509"} Sep 16 07:28:10.206435 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.206412 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" event={"ID":"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b","Type":"ContainerStarted","Data":"3ce4ba79d175a72827a09df6c6be6a1a710a80cbf772b6af8bdf79a7a12596d4"} Sep 16 07:28:10.207710 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.207684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hjlx2" event={"ID":"92f584a7-6d57-4f6b-9f18-b49f43a95e4e","Type":"ContainerStarted","Data":"97d88e7db4457fa9676a12e21c8177d2cd99cc08bf1c314e61ef6d554a6c6e3d"} Sep 16 07:28:10.209152 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.209128 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" event={"ID":"8535a87b-98d2-487c-9ed7-e9c9d4468e71","Type":"ContainerStarted","Data":"53c4026a5f6e3538804630bb7d700f258415d2e2f16dcea913f6dfb38aeed9da"} Sep 16 07:28:10.210429 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.210411 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6qkmk" event={"ID":"4b4e05ee-c901-4380-91d5-714df46eec55","Type":"ContainerStarted","Data":"9488271aa3e5b4f5fc16d4a19cbc3b8524281f2f45bd77a64d3768c7fa32a829"} Sep 16 07:28:10.211847 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.211822 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5mpgc" event={"ID":"4c658af4-a137-4b9e-9573-b66f4b836ba9","Type":"ContainerStarted","Data":"c1953a2058f620eea68fa2048704e37dd3756ec77e9711b135d21c6c7787a778"} Sep 16 07:28:10.213198 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.213178 2580 generic.go:356] "Generic (PLEG): container finished" podID="6d34d815-6ad3-4107-9499-e004e45cb6ba" containerID="97b6ab6a27e8fcf913b776d2701174716bc1e85df47efa34c0853768b44ee1cf" exitCode=0 Sep 16 07:28:10.213271 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.213234 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" event={"ID":"6d34d815-6ad3-4107-9499-e004e45cb6ba","Type":"ContainerDied","Data":"97b6ab6a27e8fcf913b776d2701174716bc1e85df47efa34c0853768b44ee1cf"} Sep 16 07:28:10.214722 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.214704 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-962g2" event={"ID":"28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2","Type":"ContainerStarted","Data":"6d43918f1fbab20999dccfb10fd7a8f49ce0ed140cfc31b0c9abe5833521f5a1"} Sep 16 07:28:10.224759 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.224724 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-65.ec2.internal" podStartSLOduration=20.224698965 podStartE2EDuration="20.224698965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:27:54.186354954 +0000 UTC m=+5.710708735" watchObservedRunningTime="2026-09-16 07:28:10.224698965 +0000 UTC m=+21.749052726" Sep 16 07:28:10.225013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.224993 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hjlx2" podStartSLOduration=3.5428294769999997 podStartE2EDuration="21.224984304s" totalImagesPullingTime="17.682154827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:10.224673736 +0000 UTC m=+21.749027514" watchObservedRunningTime="2026-09-16 07:28:10.224984304 +0000 UTC m=+21.749338082" Sep 16 07:28:10.241021 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.240990 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j6lxc" podStartSLOduration=3.5606910000000003 podStartE2EDuration="21.240979255s" totalImagesPullingTime="17.680288255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:10.240896071 +0000 UTC m=+21.765249850" watchObservedRunningTime="2026-09-16 07:28:10.240979255 +0000 UTC m=+21.765333034" Sep 16 07:28:10.254051 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.254011 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-962g2" podStartSLOduration=3.561044107 podStartE2EDuration="21.254000162s" totalImagesPullingTime="17.692956055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:10.253953956 +0000 UTC m=+21.778307757" watchObservedRunningTime="2026-09-16 07:28:10.254000162 +0000 UTC m=+21.778353965" Sep 16 07:28:10.271552 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.271520 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5mpgc" podStartSLOduration=3.50586893 podStartE2EDuration="21.271507173s" totalImagesPullingTime="17.765638243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:10.270934369 +0000 UTC m=+21.795288147" watchObservedRunningTime="2026-09-16 07:28:10.271507173 +0000 UTC m=+21.795860952" Sep 16 07:28:10.315605 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.315576 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6qkmk" podStartSLOduration=10.877225750000001 podStartE2EDuration="21.315563893s" totalImagesPullingTime="10.438338143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:10.287753771 +0000 UTC m=+21.812107551" watchObservedRunningTime="2026-09-16 07:28:10.315563893 +0000 UTC m=+21.839917671" Sep 16 07:28:10.350766 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.350748 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:10.410227 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.410204 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:10.512482 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.512453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:10.682315 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.682241 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:10.769175 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:10.769112 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:11.054673 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:11.054645 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:11.054856 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:11.054837 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:11.108951 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:11.108915 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:11.210519 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:11.210488 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:11.220862 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:11.220827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" event={"ID":"99e8b563-af63-4f93-b619-8a7e4c07603d","Type":"ContainerStarted","Data":"e3d26e7056e3da2289a5e0742c05ff566a4257fdf98af7aa6924671e3f6afa66"} Sep 16 07:28:11.220962 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:11.220912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" event={"ID":"99e8b563-af63-4f93-b619-8a7e4c07603d","Type":"ContainerStarted","Data":"59dfc7d3bc0f2a2722afef98c88136952d040545ea1abf62118b3aa9cda52c54"} Sep 16 07:28:11.220962 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:11.220957 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" event={"ID":"99e8b563-af63-4f93-b619-8a7e4c07603d","Type":"ContainerStarted","Data":"33debcf728f7059d89d8118cf5b63dbc4595add0df2571759fff616b9447f6df"} Sep 16 07:28:11.222565 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:11.222539 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qdfwd" event={"ID":"67eb3291-276f-4393-b1f0-85f4eb5b1887","Type":"ContainerStarted","Data":"ace6fd030365c56a0bbdbc2c54226310edada2217cd5fe868a9d62446615aa92"} Sep 16 07:28:11.346332 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:11.346281 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 16 07:28:11.913029 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:11.912997 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:12.013883 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:12.013765 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T07:28:11.346326938Z","UUID":"30285afc-ba15-4ab2-9fd7-a0dc2aa03a54","Handler":null,"Name":"","Endpoint":""} Sep 16 07:28:12.015514 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:12.015489 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 16 07:28:12.015634 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:12.015524 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 16 07:28:12.054484 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:12.054459 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:12.054622 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:12.054564 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:12.054622 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:12.054585 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:12.054738 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:12.054698 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:12.227630 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:12.227560 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" event={"ID":"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b","Type":"ContainerStarted","Data":"a7a824fc30f947f7816c6f1884edbdfb5095d38a97a5f86e01c0a432e5098dff"} Sep 16 07:28:12.306728 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:12.306703 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:12.407186 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:12.407163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:13.054616 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:13.054447 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:13.054873 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:13.054713 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:13.234622 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:13.234579 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" event={"ID":"99e8b563-af63-4f93-b619-8a7e4c07603d","Type":"ContainerStarted","Data":"678d12bbdff34894c89e49d8a3d45a66db8f1721450da3b50413e871c9fd44b1"} Sep 16 07:28:13.237240 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:13.237212 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" event={"ID":"06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b","Type":"ContainerStarted","Data":"c6f16c0be68799a30d8ec39022a993da9a4f66a82451e2d84ed51bd68530a5f5"} Sep 16 07:28:13.257163 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:13.255396 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b24st" podStartSLOduration=4.4293209749999995 podStartE2EDuration="24.255378252s" totalImagesPullingTime="19.826057277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:13.254898407 +0000 UTC m=+24.779252189" watchObservedRunningTime="2026-09-16 07:28:13.255378252 +0000 UTC m=+24.779732031" Sep 16 07:28:13.257163 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:13.256222 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qdfwd" podStartSLOduration=6.577184068 podStartE2EDuration="24.256208959s" totalImagesPullingTime="17.679024891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:11.2360586 +0000 UTC m=+22.760412379" watchObservedRunningTime="2026-09-16 07:28:13.256208959 +0000 UTC m=+24.780562740" Sep 16 07:28:13.480624 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:13.480550 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:28:13.481194 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:13.481166 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:28:14.054949 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:14.054917 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:14.055161 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:14.054917 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:14.055161 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:14.055031 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:14.055279 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:14.055157 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:15.055179 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:15.055019 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:15.055491 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:15.055245 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:15.245232 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:15.245180 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" event={"ID":"99e8b563-af63-4f93-b619-8a7e4c07603d","Type":"ContainerStarted","Data":"38948d283d3d362ee5b21ec3e7851dc391f8fbb58216c7540f942c48ca116f5b"} Sep 16 07:28:15.245399 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:15.245384 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:28:15.247096 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:15.247072 2580 generic.go:356] "Generic (PLEG): container finished" podID="6d34d815-6ad3-4107-9499-e004e45cb6ba" containerID="e192e1cdd5c9b79e337b588e352cd9eb592304f7dab8c5990ae87cb9392150c7" exitCode=0 Sep 16 07:28:15.247176 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:15.247112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" event={"ID":"6d34d815-6ad3-4107-9499-e004e45cb6ba","Type":"ContainerDied","Data":"e192e1cdd5c9b79e337b588e352cd9eb592304f7dab8c5990ae87cb9392150c7"} Sep 16 07:28:15.264079 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:15.264062 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:28:15.295288 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:15.295257 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" podStartSLOduration=8.577636652 podStartE2EDuration="26.295244555s" totalImagesPullingTime="17.717607903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:15.277859152 +0000 UTC m=+26.802212930" watchObservedRunningTime="2026-09-16 07:28:15.295244555 +0000 UTC m=+26.819598334" Sep 16 07:28:16.054966 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.054936 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:16.055185 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:16.055036 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:16.055185 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.055085 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:16.055185 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:16.055177 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:16.249905 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.249869 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:28:16.250107 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.249945 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:28:16.263781 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.263762 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:28:16.590553 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.590490 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-crdsq"] Sep 16 07:28:16.590673 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.590598 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:16.590736 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:16.590690 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:16.593663 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.593604 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zcss9"] Sep 16 07:28:16.593774 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.593716 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:16.593873 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:16.593813 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:16.594299 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.594278 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-99h7c"] Sep 16 07:28:16.594363 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:16.594356 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:16.594446 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:16.594429 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:17.253569 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:17.253532 2580 generic.go:356] "Generic (PLEG): container finished" podID="6d34d815-6ad3-4107-9499-e004e45cb6ba" containerID="8b38c44429c411e62636e612f35292f5aefd994379bcd4a8674c45a7e944d49f" exitCode=0 Sep 16 07:28:17.254228 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:17.253610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" event={"ID":"6d34d815-6ad3-4107-9499-e004e45cb6ba","Type":"ContainerDied","Data":"8b38c44429c411e62636e612f35292f5aefd994379bcd4a8674c45a7e944d49f"} Sep 16 07:28:18.055104 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:18.055079 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:18.055238 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:18.055079 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:18.055238 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:18.055172 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:18.055310 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:18.055266 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:18.055310 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:18.055078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:18.055374 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:18.055347 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:19.259980 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:19.259807 2580 generic.go:356] "Generic (PLEG): container finished" podID="6d34d815-6ad3-4107-9499-e004e45cb6ba" containerID="907092d487d75895599f33873eaf5ce0f496f27b7950c20fd5a1eb9672a7711d" exitCode=0 Sep 16 07:28:19.260406 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:19.259876 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" event={"ID":"6d34d815-6ad3-4107-9499-e004e45cb6ba","Type":"ContainerDied","Data":"907092d487d75895599f33873eaf5ce0f496f27b7950c20fd5a1eb9672a7711d"} Sep 16 07:28:19.770913 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:19.770883 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:28:19.771068 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:19.771006 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 16 07:28:19.771541 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:19.771515 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6qkmk" Sep 16 07:28:19.831831 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:19.831806 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:20.055073 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:20.054999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:20.055073 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:20.055019 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:20.055073 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:20.055009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:20.055303 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:20.055134 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:20.055303 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:20.055182 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:20.055303 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:20.055254 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:21.401299 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:21.401258 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:22.054508 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.054477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:22.054695 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.054477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:22.054695 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.054606 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:28:22.054821 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.054691 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crdsq" podUID="cbfcdb7d-38d2-4284-b416-750df96a620a" Sep 16 07:28:22.054821 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.054478 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:22.054907 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.054820 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zcss9" podUID="d68eaab1-3586-4f06-8c86-d66a9908733a" Sep 16 07:28:22.557543 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.557511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:22.558007 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.557667 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 07:28:22.558007 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.557725 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret podName:cbfcdb7d-38d2-4284-b416-750df96a620a nodeName:}" failed. No retries permitted until 2026-09-16 07:28:54.557710195 +0000 UTC m=+66.082063968 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret") pod "global-pull-secret-syncer-crdsq" (UID: "cbfcdb7d-38d2-4284-b416-750df96a620a") : object "kube-system"/"original-pull-secret" not registered Sep 16 07:28:22.658616 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.658549 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:22.658755 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.658676 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:28:22.658755 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.658728 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:28:54.658714636 +0000 UTC m=+66.183068406 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 07:28:22.759298 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.759262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:22.759439 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.759412 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 07:28:22.759439 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.759433 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 07:28:22.759539 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.759442 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vpln4 for pod openshift-network-diagnostics/network-check-target-zcss9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:28:22.759539 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:22.759488 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4 podName:d68eaab1-3586-4f06-8c86-d66a9908733a nodeName:}" failed. No retries permitted until 2026-09-16 07:28:54.759475378 +0000 UTC m=+66.283829135 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpln4" (UniqueName: "kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4") pod "network-check-target-zcss9" (UID: "d68eaab1-3586-4f06-8c86-d66a9908733a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 07:28:22.798536 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.798510 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-65.ec2.internal" event="NodeReady" Sep 16 07:28:22.798667 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.798643 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 16 07:28:22.840757 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.840700 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vmll6"] Sep 16 07:28:22.845373 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.845351 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zrfpb"] Sep 16 07:28:22.845495 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.845481 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:22.847810 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.847776 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 16 07:28:22.847928 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.847842 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 16 07:28:22.847928 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.847855 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8grdh\"" Sep 16 07:28:22.848680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.848663 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zrfpb" Sep 16 07:28:22.850712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.850666 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 16 07:28:22.850712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.850711 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 16 07:28:22.850896 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.850728 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 16 07:28:22.850896 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.850714 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jmsns\"" Sep 16 07:28:22.854810 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.854697 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vmll6"] Sep 16 07:28:22.855187 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.855166 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zrfpb"] Sep 16 07:28:22.953261 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.953195 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:22.961221 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.961192 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzz5t\" (UniqueName: \"kubernetes.io/projected/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-kube-api-access-bzz5t\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:22.961363 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.961248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b0aee94-ad88-4d46-93f1-01ccd630a7cd-cert\") pod \"ingress-canary-zrfpb\" (UID: \"5b0aee94-ad88-4d46-93f1-01ccd630a7cd\") " pod="openshift-ingress-canary/ingress-canary-zrfpb" Sep 16 07:28:22.961429 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.961361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:22.961429 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.961399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-config-volume\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:22.961515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.961430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spj5d\" (UniqueName: \"kubernetes.io/projected/5b0aee94-ad88-4d46-93f1-01ccd630a7cd-kube-api-access-spj5d\") pod \"ingress-canary-zrfpb\" (UID: \"5b0aee94-ad88-4d46-93f1-01ccd630a7cd\") " pod="openshift-ingress-canary/ingress-canary-zrfpb" Sep 16 07:28:22.961569 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:22.961515 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-tmp-dir\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:23.013661 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.013636 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:23.061974 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.061944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzz5t\" (UniqueName: \"kubernetes.io/projected/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-kube-api-access-bzz5t\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:23.062111 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.061997 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b0aee94-ad88-4d46-93f1-01ccd630a7cd-cert\") pod \"ingress-canary-zrfpb\" (UID: \"5b0aee94-ad88-4d46-93f1-01ccd630a7cd\") " pod="openshift-ingress-canary/ingress-canary-zrfpb" Sep 16 07:28:23.062111 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.062028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:23.062111 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.062049 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-config-volume\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:23.062111 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.062073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spj5d\" (UniqueName: \"kubernetes.io/projected/5b0aee94-ad88-4d46-93f1-01ccd630a7cd-kube-api-access-spj5d\") pod \"ingress-canary-zrfpb\" (UID: \"5b0aee94-ad88-4d46-93f1-01ccd630a7cd\") " pod="openshift-ingress-canary/ingress-canary-zrfpb" Sep 16 07:28:23.062323 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.062203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-tmp-dir\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:23.062374 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:23.062325 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 07:28:23.062427 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:23.062381 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls podName:ddf610bc-54f4-45e1-9c24-dec4ce0048b6 nodeName:}" failed. No retries permitted until 2026-09-16 07:28:23.562362053 +0000 UTC m=+35.086715826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls") pod "dns-default-vmll6" (UID: "ddf610bc-54f4-45e1-9c24-dec4ce0048b6") : secret "dns-default-metrics-tls" not found Sep 16 07:28:23.062615 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.062596 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-tmp-dir\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:23.062718 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.062691 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-config-volume\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:23.066999 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.066974 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b0aee94-ad88-4d46-93f1-01ccd630a7cd-cert\") pod \"ingress-canary-zrfpb\" (UID: \"5b0aee94-ad88-4d46-93f1-01ccd630a7cd\") " pod="openshift-ingress-canary/ingress-canary-zrfpb" Sep 16 07:28:23.070162 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.070113 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzz5t\" (UniqueName: \"kubernetes.io/projected/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-kube-api-access-bzz5t\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:23.073352 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.073332 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spj5d\" (UniqueName: \"kubernetes.io/projected/5b0aee94-ad88-4d46-93f1-01ccd630a7cd-kube-api-access-spj5d\") pod \"ingress-canary-zrfpb\" (UID: \"5b0aee94-ad88-4d46-93f1-01ccd630a7cd\") " pod="openshift-ingress-canary/ingress-canary-zrfpb" Sep 16 07:28:23.165232 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.165205 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zrfpb" Sep 16 07:28:23.565459 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:23.565422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:23.565914 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:23.565588 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 07:28:23.565914 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:23.565668 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls podName:ddf610bc-54f4-45e1-9c24-dec4ce0048b6 nodeName:}" failed. No retries permitted until 2026-09-16 07:28:24.565650698 +0000 UTC m=+36.090004460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls") pod "dns-default-vmll6" (UID: "ddf610bc-54f4-45e1-9c24-dec4ce0048b6") : secret "dns-default-metrics-tls" not found Sep 16 07:28:24.054896 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.054702 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:24.054997 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.054717 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:24.055284 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.054757 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:24.058505 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.058158 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 16 07:28:24.058505 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.058200 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 16 07:28:24.058505 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.058231 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xjqjb\"" Sep 16 07:28:24.058505 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.058202 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wbnpf\"" Sep 16 07:28:24.058505 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.058163 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 16 07:28:24.058505 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.058160 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 16 07:28:24.225059 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.225008 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zrfpb"] Sep 16 07:28:24.269613 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:28:24.269587 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0aee94_ad88_4d46_93f1_01ccd630a7cd.slice/crio-bd3b805d7152c45f15a6949cc9f35e2aa7c789b380bc2fbed07280a736393d08 WatchSource:0}: Error finding container bd3b805d7152c45f15a6949cc9f35e2aa7c789b380bc2fbed07280a736393d08: Status 404 returned error can't find the container with id bd3b805d7152c45f15a6949cc9f35e2aa7c789b380bc2fbed07280a736393d08 Sep 16 07:28:24.274332 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.274273 2580 generic.go:356] "Generic (PLEG): container finished" podID="6d34d815-6ad3-4107-9499-e004e45cb6ba" containerID="b0e646afe9fe9bd4eb97f8792fad633ede6897d2145e67dbb9724dd1cb63448b" exitCode=0 Sep 16 07:28:24.274383 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.274329 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" event={"ID":"6d34d815-6ad3-4107-9499-e004e45cb6ba","Type":"ContainerDied","Data":"b0e646afe9fe9bd4eb97f8792fad633ede6897d2145e67dbb9724dd1cb63448b"} Sep 16 07:28:24.571641 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:24.571567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:24.572213 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:24.571717 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 07:28:24.572213 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:24.571780 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls podName:ddf610bc-54f4-45e1-9c24-dec4ce0048b6 nodeName:}" failed. No retries permitted until 2026-09-16 07:28:26.571762201 +0000 UTC m=+38.096115969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls") pod "dns-default-vmll6" (UID: "ddf610bc-54f4-45e1-9c24-dec4ce0048b6") : secret "dns-default-metrics-tls" not found Sep 16 07:28:25.280553 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:25.280520 2580 generic.go:356] "Generic (PLEG): container finished" podID="6d34d815-6ad3-4107-9499-e004e45cb6ba" containerID="960a3fdef1124ceccdde237efeab152b65b81ec8fdb7c033c235b79b5cd626db" exitCode=0 Sep 16 07:28:25.280721 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:25.280602 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" event={"ID":"6d34d815-6ad3-4107-9499-e004e45cb6ba","Type":"ContainerDied","Data":"960a3fdef1124ceccdde237efeab152b65b81ec8fdb7c033c235b79b5cd626db"} Sep 16 07:28:25.281995 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:25.281971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zrfpb" event={"ID":"5b0aee94-ad88-4d46-93f1-01ccd630a7cd","Type":"ContainerStarted","Data":"bd3b805d7152c45f15a6949cc9f35e2aa7c789b380bc2fbed07280a736393d08"} Sep 16 07:28:26.287677 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:26.287483 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" event={"ID":"6d34d815-6ad3-4107-9499-e004e45cb6ba","Type":"ContainerStarted","Data":"3309ad989f08e7a6d1fa151447525966e59dc6ee6ad90f31b55e0f492608f439"} Sep 16 07:28:26.309330 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:26.309285 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mq8vr" podStartSLOduration=8.058984791 podStartE2EDuration="37.309272081s" totalImagesPullingTime="29.25028729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:26.307845936 +0000 UTC m=+37.832199706" watchObservedRunningTime="2026-09-16 07:28:26.309272081 +0000 UTC m=+37.833625855" Sep 16 07:28:26.433312 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:26.433249 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:26.495340 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:26.495317 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:26.586831 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:26.586796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:26.586932 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:26.586915 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 07:28:26.586979 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:26.586973 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls podName:ddf610bc-54f4-45e1-9c24-dec4ce0048b6 nodeName:}" failed. No retries permitted until 2026-09-16 07:28:30.586958807 +0000 UTC m=+42.111312563 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls") pod "dns-default-vmll6" (UID: "ddf610bc-54f4-45e1-9c24-dec4ce0048b6") : secret "dns-default-metrics-tls" not found Sep 16 07:28:26.625984 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:26.625961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:26.699888 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:26.699864 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:26.959427 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:26.959355 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:27.290805 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:27.290721 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zrfpb" event={"ID":"5b0aee94-ad88-4d46-93f1-01ccd630a7cd","Type":"ContainerStarted","Data":"12333ee01691059ad7fca206e9c5419b8f8594c1961541f79805e4dea0bd61d6"} Sep 16 07:28:27.305427 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:27.305383 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zrfpb" podStartSLOduration=2.979181376 podStartE2EDuration="5.305368539s" totalImagesPullingTime="2.326187163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:28:27.305027661 +0000 UTC m=+38.829381441" watchObservedRunningTime="2026-09-16 07:28:27.305368539 +0000 UTC m=+38.829722348" Sep 16 07:28:27.820657 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:27.820627 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:27.888922 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:27.888896 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:30.179317 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:30.179280 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:30.238733 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:30.238694 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:30.612005 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:30.611973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:30.612159 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:30.612075 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 07:28:30.612159 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:30.612139 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls podName:ddf610bc-54f4-45e1-9c24-dec4ce0048b6 nodeName:}" failed. No retries permitted until 2026-09-16 07:28:38.612124379 +0000 UTC m=+50.136478135 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls") pod "dns-default-vmll6" (UID: "ddf610bc-54f4-45e1-9c24-dec4ce0048b6") : secret "dns-default-metrics-tls" not found Sep 16 07:28:38.661803 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:38.661753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:38.662208 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:38.661896 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 07:28:38.662208 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:38.661968 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls podName:ddf610bc-54f4-45e1-9c24-dec4ce0048b6 nodeName:}" failed. No retries permitted until 2026-09-16 07:28:54.661953666 +0000 UTC m=+66.186307423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls") pod "dns-default-vmll6" (UID: "ddf610bc-54f4-45e1-9c24-dec4ce0048b6") : secret "dns-default-metrics-tls" not found Sep 16 07:28:48.271744 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:48.271716 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9rxmw" Sep 16 07:28:48.420070 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:48.420038 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:48.473595 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:48.473557 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:48.781501 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:48.781470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:48.881976 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:48.881945 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:28:54.560367 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.560332 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:54.562717 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.562700 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 16 07:28:54.574690 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.574666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cbfcdb7d-38d2-4284-b416-750df96a620a-original-pull-secret\") pod \"global-pull-secret-syncer-crdsq\" (UID: \"cbfcdb7d-38d2-4284-b416-750df96a620a\") " pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:54.660961 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.660933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:28:54.663210 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.663194 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 16 07:28:54.671986 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:54.671969 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 16 07:28:54.672043 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:54.672020 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:29:58.672005181 +0000 UTC m=+130.196358938 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : secret "metrics-daemon-secret" not found Sep 16 07:28:54.688440 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.688425 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crdsq" Sep 16 07:28:54.762662 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.762240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:28:54.762662 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.762313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:54.762662 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:54.762488 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 07:28:54.762662 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:28:54.762595 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls podName:ddf610bc-54f4-45e1-9c24-dec4ce0048b6 nodeName:}" failed. No retries permitted until 2026-09-16 07:29:26.762558834 +0000 UTC m=+98.286912596 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls") pod "dns-default-vmll6" (UID: "ddf610bc-54f4-45e1-9c24-dec4ce0048b6") : secret "dns-default-metrics-tls" not found Sep 16 07:28:54.765806 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.765765 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 16 07:28:54.775246 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.775229 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 16 07:28:54.786320 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.786300 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpln4\" (UniqueName: \"kubernetes.io/projected/d68eaab1-3586-4f06-8c86-d66a9908733a-kube-api-access-vpln4\") pod \"network-check-target-zcss9\" (UID: \"d68eaab1-3586-4f06-8c86-d66a9908733a\") " pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:54.985163 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.985141 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-crdsq"] Sep 16 07:28:54.988530 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:28:54.988510 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbfcdb7d_38d2_4284_b416_750df96a620a.slice/crio-32680f5941c17882fad17174c1ecbf6df58d291a412c33b466f30e902a5e543e WatchSource:0}: Error finding container 32680f5941c17882fad17174c1ecbf6df58d291a412c33b466f30e902a5e543e: Status 404 returned error can't find the container with id 32680f5941c17882fad17174c1ecbf6df58d291a412c33b466f30e902a5e543e Sep 16 07:28:54.997676 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:54.997656 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wbnpf\"" Sep 16 07:28:55.006174 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:55.006157 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:28:55.290812 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:55.290735 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zcss9"] Sep 16 07:28:55.293617 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:28:55.293594 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68eaab1_3586_4f06_8c86_d66a9908733a.slice/crio-0fec7b3cc6eaea9787c7c7a5816ba4c33cf01fe90a1a5dcaf07ba03ea338d1af WatchSource:0}: Error finding container 0fec7b3cc6eaea9787c7c7a5816ba4c33cf01fe90a1a5dcaf07ba03ea338d1af: Status 404 returned error can't find the container with id 0fec7b3cc6eaea9787c7c7a5816ba4c33cf01fe90a1a5dcaf07ba03ea338d1af Sep 16 07:28:55.351533 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:55.351499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zcss9" event={"ID":"d68eaab1-3586-4f06-8c86-d66a9908733a","Type":"ContainerStarted","Data":"0fec7b3cc6eaea9787c7c7a5816ba4c33cf01fe90a1a5dcaf07ba03ea338d1af"} Sep 16 07:28:55.352583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:28:55.352563 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-crdsq" event={"ID":"cbfcdb7d-38d2-4284-b416-750df96a620a","Type":"ContainerStarted","Data":"32680f5941c17882fad17174c1ecbf6df58d291a412c33b466f30e902a5e543e"} Sep 16 07:29:00.368617 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:00.368582 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zcss9" event={"ID":"d68eaab1-3586-4f06-8c86-d66a9908733a","Type":"ContainerStarted","Data":"944bb6dbfa70deed8889f710c3280f4efead8d459ef0c37a4c59ded476aac389"} Sep 16 07:29:00.369156 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:00.368692 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:29:00.370157 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:00.370136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-crdsq" event={"ID":"cbfcdb7d-38d2-4284-b416-750df96a620a","Type":"ContainerStarted","Data":"8abfbf5a5d3c9ee1681d369b2a23af6e06617473e9cea3f8472b0eb91c910049"} Sep 16 07:29:00.382731 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:00.382691 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zcss9" podStartSLOduration=67.104983978 podStartE2EDuration="1m11.38267647s" totalImagesPullingTime="4.277692492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:29:00.382196378 +0000 UTC m=+71.906550160" watchObservedRunningTime="2026-09-16 07:29:00.38267647 +0000 UTC m=+71.907030253" Sep 16 07:29:00.395782 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:00.395746 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-crdsq" podStartSLOduration=66.80980504 podStartE2EDuration="1m11.395734067s" totalImagesPullingTime="4.585929027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:29:00.395239509 +0000 UTC m=+71.919593289" watchObservedRunningTime="2026-09-16 07:29:00.395734067 +0000 UTC m=+71.920087842" Sep 16 07:29:14.750575 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:14.750469 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:29:14.802461 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:14.802438 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:29:26.050451 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.050413 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6"] Sep 16 07:29:26.053546 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.053530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.058213 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.058193 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 16 07:29:26.059088 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.059069 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 16 07:29:26.059190 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.059080 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 16 07:29:26.061191 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.061156 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 16 07:29:26.083995 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.083971 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6"] Sep 16 07:29:26.152556 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.152528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c9gq\" (UniqueName: \"kubernetes.io/projected/6059d506-64cd-4111-bf5b-0655fee4b1d4-kube-api-access-6c9gq\") pod \"klusterlet-addon-workmgr-676f59f68c-v2kz6\" (UID: \"6059d506-64cd-4111-bf5b-0655fee4b1d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.152643 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.152560 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6059d506-64cd-4111-bf5b-0655fee4b1d4-klusterlet-config\") pod \"klusterlet-addon-workmgr-676f59f68c-v2kz6\" (UID: \"6059d506-64cd-4111-bf5b-0655fee4b1d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.152643 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.152578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6059d506-64cd-4111-bf5b-0655fee4b1d4-tmp\") pod \"klusterlet-addon-workmgr-676f59f68c-v2kz6\" (UID: \"6059d506-64cd-4111-bf5b-0655fee4b1d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.253531 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.253506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c9gq\" (UniqueName: \"kubernetes.io/projected/6059d506-64cd-4111-bf5b-0655fee4b1d4-kube-api-access-6c9gq\") pod \"klusterlet-addon-workmgr-676f59f68c-v2kz6\" (UID: \"6059d506-64cd-4111-bf5b-0655fee4b1d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.253646 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.253549 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6059d506-64cd-4111-bf5b-0655fee4b1d4-klusterlet-config\") pod \"klusterlet-addon-workmgr-676f59f68c-v2kz6\" (UID: \"6059d506-64cd-4111-bf5b-0655fee4b1d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.253646 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.253576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6059d506-64cd-4111-bf5b-0655fee4b1d4-tmp\") pod \"klusterlet-addon-workmgr-676f59f68c-v2kz6\" (UID: \"6059d506-64cd-4111-bf5b-0655fee4b1d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.254049 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.254025 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6059d506-64cd-4111-bf5b-0655fee4b1d4-tmp\") pod \"klusterlet-addon-workmgr-676f59f68c-v2kz6\" (UID: \"6059d506-64cd-4111-bf5b-0655fee4b1d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.255848 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.255830 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6059d506-64cd-4111-bf5b-0655fee4b1d4-klusterlet-config\") pod \"klusterlet-addon-workmgr-676f59f68c-v2kz6\" (UID: \"6059d506-64cd-4111-bf5b-0655fee4b1d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.261231 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.261209 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c9gq\" (UniqueName: \"kubernetes.io/projected/6059d506-64cd-4111-bf5b-0655fee4b1d4-kube-api-access-6c9gq\") pod \"klusterlet-addon-workmgr-676f59f68c-v2kz6\" (UID: \"6059d506-64cd-4111-bf5b-0655fee4b1d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.362516 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.362492 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:26.627883 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.627813 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6"] Sep 16 07:29:26.630395 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:29:26.630365 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6059d506_64cd_4111_bf5b_0655fee4b1d4.slice/crio-2072ac613a33ff7e8ab3a4bb4fd2123ad8efdc588997662dc098ad99d7d97445 WatchSource:0}: Error finding container 2072ac613a33ff7e8ab3a4bb4fd2123ad8efdc588997662dc098ad99d7d97445: Status 404 returned error can't find the container with id 2072ac613a33ff7e8ab3a4bb4fd2123ad8efdc588997662dc098ad99d7d97445 Sep 16 07:29:26.857618 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:26.857580 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:29:26.857739 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:29:26.857619 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 07:29:26.857739 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:29:26.857684 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls podName:ddf610bc-54f4-45e1-9c24-dec4ce0048b6 nodeName:}" failed. No retries permitted until 2026-09-16 07:30:30.857668581 +0000 UTC m=+162.382022342 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls") pod "dns-default-vmll6" (UID: "ddf610bc-54f4-45e1-9c24-dec4ce0048b6") : secret "dns-default-metrics-tls" not found Sep 16 07:29:27.431776 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:27.431744 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" event={"ID":"6059d506-64cd-4111-bf5b-0655fee4b1d4","Type":"ContainerStarted","Data":"2072ac613a33ff7e8ab3a4bb4fd2123ad8efdc588997662dc098ad99d7d97445"} Sep 16 07:29:31.374006 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:31.373984 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zcss9" Sep 16 07:29:31.446138 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:31.446068 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" event={"ID":"6059d506-64cd-4111-bf5b-0655fee4b1d4","Type":"ContainerStarted","Data":"2bd20fc957025487a5557a7015503f18f970e48aa0a3b4f2c1834015ce388846"} Sep 16 07:29:31.446273 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:31.446261 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:31.447602 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:31.447581 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:29:31.460828 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:31.460772 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" podStartSLOduration=0.945653768 podStartE2EDuration="5.460759281s" totalImagesPullingTime="4.515105513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:29:31.459669685 +0000 UTC m=+102.984023463" watchObservedRunningTime="2026-09-16 07:29:31.460759281 +0000 UTC m=+102.985113060" Sep 16 07:29:52.521992 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:52.521962 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:29:58.752759 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:29:58.752715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:29:58.753163 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:29:58.752873 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 16 07:29:58.753163 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:29:58.752935 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:32:00.752918777 +0000 UTC m=+252.277272534 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : secret "metrics-daemon-secret" not found Sep 16 07:30:25.858795 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:30:25.858743 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vmll6" podUID="ddf610bc-54f4-45e1-9c24-dec4ce0048b6" Sep 16 07:30:26.576641 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:30:26.576570 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmll6" Sep 16 07:30:27.066526 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:30:27.066490 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:30:30.859802 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:30:30.859741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:30:30.860242 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:30:30.859904 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 07:30:30.860242 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:30:30.859977 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls podName:ddf610bc-54f4-45e1-9c24-dec4ce0048b6 nodeName:}" failed. No retries permitted until 2026-09-16 07:32:32.85996135 +0000 UTC m=+284.384315124 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls") pod "dns-default-vmll6" (UID: "ddf610bc-54f4-45e1-9c24-dec4ce0048b6") : secret "dns-default-metrics-tls" not found Sep 16 07:30:42.054526 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:30:42.054447 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:31:26.886908 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:26.886865 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:31.446609 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:31.446565 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" podUID="6059d506-64cd-4111-bf5b-0655fee4b1d4" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused" Sep 16 07:31:31.728979 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:31.728940 2580 generic.go:356] "Generic (PLEG): container finished" podID="6059d506-64cd-4111-bf5b-0655fee4b1d4" containerID="2bd20fc957025487a5557a7015503f18f970e48aa0a3b4f2c1834015ce388846" exitCode=1 Sep 16 07:31:31.729127 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:31.728999 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" event={"ID":"6059d506-64cd-4111-bf5b-0655fee4b1d4","Type":"ContainerDied","Data":"2bd20fc957025487a5557a7015503f18f970e48aa0a3b4f2c1834015ce388846"} Sep 16 07:31:31.729275 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:31.729263 2580 scope.go:117] "RemoveContainer" containerID="2bd20fc957025487a5557a7015503f18f970e48aa0a3b4f2c1834015ce388846" Sep 16 07:31:32.732755 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:32.732722 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" event={"ID":"6059d506-64cd-4111-bf5b-0655fee4b1d4","Type":"ContainerStarted","Data":"ba967960f05f03304d5b8d058c5d0d48311e89170d765946126a4b8a9a696b4e"} Sep 16 07:31:32.733119 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:32.733011 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:31:32.733508 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:32.733488 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-676f59f68c-v2kz6" Sep 16 07:31:53.645641 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.645599 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg"] Sep 16 07:31:53.648586 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.648565 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg" Sep 16 07:31:53.650882 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.650864 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-hztql\"" Sep 16 07:31:53.661034 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.661016 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg"] Sep 16 07:31:53.744708 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.744679 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv"] Sep 16 07:31:53.747361 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.747346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:31:53.752326 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.752308 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 16 07:31:53.752424 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.752393 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9vq65\"" Sep 16 07:31:53.752483 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.752463 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 16 07:31:53.752777 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.752763 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 16 07:31:53.753449 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.753431 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp"] Sep 16 07:31:53.755907 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.755892 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59548f6486-jrsg2"] Sep 16 07:31:53.756085 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.756008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:53.758454 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.758439 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-bhpfr"] Sep 16 07:31:53.758582 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.758565 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.759150 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.759132 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 16 07:31:53.759339 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.759315 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 16 07:31:53.759467 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.759249 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 16 07:31:53.759467 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.759375 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 16 07:31:53.759467 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.759246 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hqfs6\"" Sep 16 07:31:53.760556 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.760539 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 16 07:31:53.760649 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.760567 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b6zs2\"" Sep 16 07:31:53.761218 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.761187 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv"] Sep 16 07:31:53.761313 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.761274 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.761830 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.761812 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 16 07:31:53.761939 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.761815 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 16 07:31:53.763538 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.763520 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fmccz\"" Sep 16 07:31:53.763950 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.763767 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 16 07:31:53.763950 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.763841 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 16 07:31:53.763950 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.763856 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 16 07:31:53.764178 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.764134 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 16 07:31:53.767744 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.767725 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 16 07:31:53.769466 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.769443 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp"] Sep 16 07:31:53.769956 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.769916 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 16 07:31:53.770493 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.770474 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-bhpfr"] Sep 16 07:31:53.776182 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.776160 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59548f6486-jrsg2"] Sep 16 07:31:53.785679 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.785660 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65fsk\" (UniqueName: \"kubernetes.io/projected/6c1969b6-ef6f-4527-ad67-3dd392558050-kube-api-access-65fsk\") pod \"network-check-source-6f8b768455-qhcvg\" (UID: \"6c1969b6-ef6f-4527-ad67-3dd392558050\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg" Sep 16 07:31:53.886611 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84595af1-2295-41ac-abe5-c448a3116f98-trusted-ca\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.886705 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886615 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjrgz\" (UniqueName: \"kubernetes.io/projected/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-kube-api-access-hjrgz\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:31:53.886705 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-image-registry-private-configuration\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.886705 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886664 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:31:53.886819 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec53573a-703d-4e8e-ab3c-05f3db9253c0-ca-trust-extracted\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.886819 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886731 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-trusted-ca\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.886819 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886765 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5klq\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-kube-api-access-t5klq\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.886916 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-bound-sa-token\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.886916 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84595af1-2295-41ac-abe5-c448a3116f98-config\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.886916 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8rdr\" (UniqueName: \"kubernetes.io/projected/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-kube-api-access-t8rdr\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:53.886916 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886889 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:53.887032 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886945 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjcz6\" (UniqueName: \"kubernetes.io/projected/84595af1-2295-41ac-abe5-c448a3116f98-kube-api-access-cjcz6\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.887032 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-installation-pull-secrets\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.887032 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.886987 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:53.887032 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.887030 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84595af1-2295-41ac-abe5-c448a3116f98-serving-cert\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.887202 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.887048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65fsk\" (UniqueName: \"kubernetes.io/projected/6c1969b6-ef6f-4527-ad67-3dd392558050-kube-api-access-65fsk\") pod \"network-check-source-6f8b768455-qhcvg\" (UID: \"6c1969b6-ef6f-4527-ad67-3dd392558050\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg" Sep 16 07:31:53.887202 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.887063 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.887202 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.887079 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-certificates\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.894606 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.894590 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65fsk\" (UniqueName: \"kubernetes.io/projected/6c1969b6-ef6f-4527-ad67-3dd392558050-kube-api-access-65fsk\") pod \"network-check-source-6f8b768455-qhcvg\" (UID: \"6c1969b6-ef6f-4527-ad67-3dd392558050\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg" Sep 16 07:31:53.908247 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.908181 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:53.958022 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.957999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg" Sep 16 07:31:53.987885 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.987860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-installation-pull-secrets\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.988012 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.987892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:53.988012 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.987921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84595af1-2295-41ac-abe5-c448a3116f98-serving-cert\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.988012 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.987947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.988178 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:53.988025 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 07:31:53.988178 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:53.988040 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59548f6486-jrsg2: secret "image-registry-tls" not found Sep 16 07:31:53.988178 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:53.988092 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls podName:ec53573a-703d-4e8e-ab3c-05f3db9253c0 nodeName:}" failed. No retries permitted until 2026-09-16 07:31:54.488073856 +0000 UTC m=+246.012427627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls") pod "image-registry-59548f6486-jrsg2" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0") : secret "image-registry-tls" not found Sep 16 07:31:53.988178 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:53.988026 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 07:31:53.988178 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-certificates\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.988178 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84595af1-2295-41ac-abe5-c448a3116f98-trusted-ca\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.988494 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:53.988187 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls podName:4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc nodeName:}" failed. No retries permitted until 2026-09-16 07:31:54.48816772 +0000 UTC m=+246.012521484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-24djp" (UID: "4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc") : secret "cluster-monitoring-operator-tls" not found Sep 16 07:31:53.988494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjrgz\" (UniqueName: \"kubernetes.io/projected/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-kube-api-access-hjrgz\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:31:53.988494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-image-registry-private-configuration\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.988494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:31:53.988494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec53573a-703d-4e8e-ab3c-05f3db9253c0-ca-trust-extracted\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.988494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988377 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-trusted-ca\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.988494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5klq\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-kube-api-access-t5klq\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.988494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-bound-sa-token\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.988494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84595af1-2295-41ac-abe5-c448a3116f98-config\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.988987 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8rdr\" (UniqueName: \"kubernetes.io/projected/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-kube-api-access-t8rdr\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:53.988987 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:53.988987 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjcz6\" (UniqueName: \"kubernetes.io/projected/84595af1-2295-41ac-abe5-c448a3116f98-kube-api-access-cjcz6\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.988987 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.988731 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-certificates\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.988987 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:53.988778 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 07:31:53.988987 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:53.988847 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls podName:b8e9d14a-fed3-497f-aa3d-d7c8e783fa01 nodeName:}" failed. No retries permitted until 2026-09-16 07:31:54.488831713 +0000 UTC m=+246.013185486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-tj6dv" (UID: "b8e9d14a-fed3-497f-aa3d-d7c8e783fa01") : secret "samples-operator-tls" not found Sep 16 07:31:53.989312 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.989036 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec53573a-703d-4e8e-ab3c-05f3db9253c0-ca-trust-extracted\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.989312 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.989235 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84595af1-2295-41ac-abe5-c448a3116f98-trusted-ca\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.989661 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.989633 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84595af1-2295-41ac-abe5-c448a3116f98-config\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.989887 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.989862 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-trusted-ca\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.989968 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.989899 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:53.990642 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.990620 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-installation-pull-secrets\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.991397 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.991373 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84595af1-2295-41ac-abe5-c448a3116f98-serving-cert\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.991461 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.991441 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-image-registry-private-configuration\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.995833 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.995650 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:53.997466 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.997441 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8rdr\" (UniqueName: \"kubernetes.io/projected/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-kube-api-access-t8rdr\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:53.998068 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.998049 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-bound-sa-token\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.998210 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.998191 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5klq\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-kube-api-access-t5klq\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:53.998622 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.998606 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjcz6\" (UniqueName: \"kubernetes.io/projected/84595af1-2295-41ac-abe5-c448a3116f98-kube-api-access-cjcz6\") pod \"console-operator-696884fb9c-bhpfr\" (UID: \"84595af1-2295-41ac-abe5-c448a3116f98\") " pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:53.999395 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:53.999343 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjrgz\" (UniqueName: \"kubernetes.io/projected/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-kube-api-access-hjrgz\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:31:54.081310 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.081270 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:31:54.225868 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.225774 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg"] Sep 16 07:31:54.229556 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:31:54.229517 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1969b6_ef6f_4527_ad67_3dd392558050.slice/crio-09144218e22c7c53bf0fe8e6a87ad664072964fbbb4e5a78181f86dda999d356 WatchSource:0}: Error finding container 09144218e22c7c53bf0fe8e6a87ad664072964fbbb4e5a78181f86dda999d356: Status 404 returned error can't find the container with id 09144218e22c7c53bf0fe8e6a87ad664072964fbbb4e5a78181f86dda999d356 Sep 16 07:31:54.349716 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.349691 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-bhpfr"] Sep 16 07:31:54.352428 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:31:54.352404 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84595af1_2295_41ac_abe5_c448a3116f98.slice/crio-0140cd27103f596c01dd5e9308f58bde6aba28bdda739eb38ccf9bce8731a8d6 WatchSource:0}: Error finding container 0140cd27103f596c01dd5e9308f58bde6aba28bdda739eb38ccf9bce8731a8d6: Status 404 returned error can't find the container with id 0140cd27103f596c01dd5e9308f58bde6aba28bdda739eb38ccf9bce8731a8d6 Sep 16 07:31:54.491938 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.491866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:54.491938 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.491916 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:54.492105 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.491950 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:31:54.492105 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:54.492019 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 07:31:54.492105 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:54.492045 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 07:31:54.492105 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:54.492083 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls podName:4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc nodeName:}" failed. No retries permitted until 2026-09-16 07:31:55.49206479 +0000 UTC m=+247.016418565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-24djp" (UID: "4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc") : secret "cluster-monitoring-operator-tls" not found Sep 16 07:31:54.492105 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:54.492092 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 07:31:54.492280 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:54.492111 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59548f6486-jrsg2: secret "image-registry-tls" not found Sep 16 07:31:54.492280 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:54.492098 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls podName:b8e9d14a-fed3-497f-aa3d-d7c8e783fa01 nodeName:}" failed. No retries permitted until 2026-09-16 07:31:55.492092485 +0000 UTC m=+247.016446242 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-tj6dv" (UID: "b8e9d14a-fed3-497f-aa3d-d7c8e783fa01") : secret "samples-operator-tls" not found Sep 16 07:31:54.492280 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:54.492178 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls podName:ec53573a-703d-4e8e-ab3c-05f3db9253c0 nodeName:}" failed. No retries permitted until 2026-09-16 07:31:55.49215443 +0000 UTC m=+247.016508199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls") pod "image-registry-59548f6486-jrsg2" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0") : secret "image-registry-tls" not found Sep 16 07:31:54.789398 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.789327 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg" event={"ID":"6c1969b6-ef6f-4527-ad67-3dd392558050","Type":"ContainerStarted","Data":"bd84335086395a45c9580c2f7079408d6a09be934200397e8e17a138e2299e84"} Sep 16 07:31:54.789398 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.789378 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg" event={"ID":"6c1969b6-ef6f-4527-ad67-3dd392558050","Type":"ContainerStarted","Data":"09144218e22c7c53bf0fe8e6a87ad664072964fbbb4e5a78181f86dda999d356"} Sep 16 07:31:54.790500 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.790471 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" event={"ID":"84595af1-2295-41ac-abe5-c448a3116f98","Type":"ContainerStarted","Data":"0140cd27103f596c01dd5e9308f58bde6aba28bdda739eb38ccf9bce8731a8d6"} Sep 16 07:31:54.803289 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:54.803246 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qhcvg" podStartSLOduration=1.80323293 podStartE2EDuration="1.80323293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:31:54.802466948 +0000 UTC m=+246.326820741" watchObservedRunningTime="2026-09-16 07:31:54.80323293 +0000 UTC m=+246.327586709" Sep 16 07:31:55.500156 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:55.500123 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:55.500351 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:55.500174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:31:55.500351 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:55.500276 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 07:31:55.500351 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:55.500290 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 07:31:55.500351 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:55.500311 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59548f6486-jrsg2: secret "image-registry-tls" not found Sep 16 07:31:55.500351 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:55.500341 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls podName:b8e9d14a-fed3-497f-aa3d-d7c8e783fa01 nodeName:}" failed. No retries permitted until 2026-09-16 07:31:57.500320928 +0000 UTC m=+249.024674699 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-tj6dv" (UID: "b8e9d14a-fed3-497f-aa3d-d7c8e783fa01") : secret "samples-operator-tls" not found Sep 16 07:31:55.500606 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:55.500366 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls podName:ec53573a-703d-4e8e-ab3c-05f3db9253c0 nodeName:}" failed. No retries permitted until 2026-09-16 07:31:57.500348073 +0000 UTC m=+249.024701851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls") pod "image-registry-59548f6486-jrsg2" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0") : secret "image-registry-tls" not found Sep 16 07:31:55.500606 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:55.500436 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:55.500606 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:55.500568 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 07:31:55.500767 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:55.500648 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls podName:4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc nodeName:}" failed. No retries permitted until 2026-09-16 07:31:57.500626802 +0000 UTC m=+249.024980562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-24djp" (UID: "4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc") : secret "cluster-monitoring-operator-tls" not found Sep 16 07:31:56.697375 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:56.697307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:56.752453 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:56.752424 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:56.798002 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:56.797982 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/0.log" Sep 16 07:31:56.798105 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:56.798014 2580 generic.go:356] "Generic (PLEG): container finished" podID="84595af1-2295-41ac-abe5-c448a3116f98" containerID="6a0f4a52f2fa1e5d2c318e1a91f2121537dd6cf92ee99ba508c73ea99d9aa37f" exitCode=255 Sep 16 07:31:56.798105 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:56.798043 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" event={"ID":"84595af1-2295-41ac-abe5-c448a3116f98","Type":"ContainerDied","Data":"6a0f4a52f2fa1e5d2c318e1a91f2121537dd6cf92ee99ba508c73ea99d9aa37f"} Sep 16 07:31:56.798281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:56.798266 2580 scope.go:117] "RemoveContainer" containerID="6a0f4a52f2fa1e5d2c318e1a91f2121537dd6cf92ee99ba508c73ea99d9aa37f" Sep 16 07:31:57.518077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:57.518035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:31:57.518259 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:57.518089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:31:57.518259 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:57.518121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:31:57.518259 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:57.518168 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 07:31:57.518259 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:57.518197 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 07:31:57.518259 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:57.518205 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 07:31:57.518259 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:57.518224 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59548f6486-jrsg2: secret "image-registry-tls" not found Sep 16 07:31:57.518259 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:57.518231 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls podName:4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc nodeName:}" failed. No retries permitted until 2026-09-16 07:32:01.518214942 +0000 UTC m=+253.042568721 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-24djp" (UID: "4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc") : secret "cluster-monitoring-operator-tls" not found Sep 16 07:31:57.518259 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:57.518245 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls podName:b8e9d14a-fed3-497f-aa3d-d7c8e783fa01 nodeName:}" failed. No retries permitted until 2026-09-16 07:32:01.518238884 +0000 UTC m=+253.042592641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-tj6dv" (UID: "b8e9d14a-fed3-497f-aa3d-d7c8e783fa01") : secret "samples-operator-tls" not found Sep 16 07:31:57.518259 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:57.518267 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls podName:ec53573a-703d-4e8e-ab3c-05f3db9253c0 nodeName:}" failed. No retries permitted until 2026-09-16 07:32:01.518250891 +0000 UTC m=+253.042604649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls") pod "image-registry-59548f6486-jrsg2" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0") : secret "image-registry-tls" not found Sep 16 07:31:57.804627 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:57.804556 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:31:57.805041 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:57.805004 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/0.log" Sep 16 07:31:57.805041 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:57.805035 2580 generic.go:356] "Generic (PLEG): container finished" podID="84595af1-2295-41ac-abe5-c448a3116f98" containerID="ae22d57d0f3c26eb1c44e5939d8584d308d994cf78fa6fca141ff737d8e51c7d" exitCode=255 Sep 16 07:31:57.805110 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:57.805093 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" event={"ID":"84595af1-2295-41ac-abe5-c448a3116f98","Type":"ContainerDied","Data":"ae22d57d0f3c26eb1c44e5939d8584d308d994cf78fa6fca141ff737d8e51c7d"} Sep 16 07:31:57.805158 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:57.805147 2580 scope.go:117] "RemoveContainer" containerID="6a0f4a52f2fa1e5d2c318e1a91f2121537dd6cf92ee99ba508c73ea99d9aa37f" Sep 16 07:31:57.805344 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:57.805319 2580 scope.go:117] "RemoveContainer" containerID="ae22d57d0f3c26eb1c44e5939d8584d308d994cf78fa6fca141ff737d8e51c7d" Sep 16 07:31:57.805527 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:57.805509 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-bhpfr_openshift-console-operator(84595af1-2295-41ac-abe5-c448a3116f98)\"" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" podUID="84595af1-2295-41ac-abe5-c448a3116f98" Sep 16 07:31:58.809419 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:58.809385 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:31:58.809926 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:58.809658 2580 scope.go:117] "RemoveContainer" containerID="ae22d57d0f3c26eb1c44e5939d8584d308d994cf78fa6fca141ff737d8e51c7d" Sep 16 07:31:58.809926 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:58.809781 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-bhpfr_openshift-console-operator(84595af1-2295-41ac-abe5-c448a3116f98)\"" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" podUID="84595af1-2295-41ac-abe5-c448a3116f98" Sep 16 07:31:59.030269 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.030236 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:59.044803 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.044767 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2gjlc"] Sep 16 07:31:59.048936 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.048918 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.051063 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.051046 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 16 07:31:59.051193 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.051176 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 16 07:31:59.051262 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.051069 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qw2t8\"" Sep 16 07:31:59.051318 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.051274 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 16 07:31:59.053657 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.053638 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 16 07:31:59.060846 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.060803 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2gjlc"] Sep 16 07:31:59.103745 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.103727 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:59.231008 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.230984 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:59.232849 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.232831 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.232918 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.232868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqqgx\" (UniqueName: \"kubernetes.io/projected/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-api-access-qqqgx\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.232963 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.232949 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2c9e87ea-05bc-40ce-ad08-37a06216c67c-crio-socket\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.233014 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.232971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.233061 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.233017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c9e87ea-05bc-40ce-ad08-37a06216c67c-data-volume\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.306567 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.306543 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:59.333548 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.333500 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c9e87ea-05bc-40ce-ad08-37a06216c67c-data-volume\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.333548 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.333541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.333654 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.333568 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqqgx\" (UniqueName: \"kubernetes.io/projected/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-api-access-qqqgx\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.333654 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.333604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2c9e87ea-05bc-40ce-ad08-37a06216c67c-crio-socket\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.333654 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.333618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.333654 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:59.333642 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 07:31:59.333807 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:59.333676 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 07:31:59.333807 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:59.333707 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:31:59.83369037 +0000 UTC m=+251.358044127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : secret "insights-runtime-extractor-tls" not found Sep 16 07:31:59.333807 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.333713 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2c9e87ea-05bc-40ce-ad08-37a06216c67c-crio-socket\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.333807 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:59.333732 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:31:59.833716439 +0000 UTC m=+251.358070214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : configmap "kube-rbac-proxy" not found Sep 16 07:31:59.333975 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.333812 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c9e87ea-05bc-40ce-ad08-37a06216c67c-data-volume\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.343931 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.343913 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqqgx\" (UniqueName: \"kubernetes.io/projected/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-api-access-qqqgx\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.406680 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.406655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:59.473068 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.473046 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:59.621086 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.621059 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:59.669289 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.669268 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:31:59.837554 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.837533 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.837869 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:31:59.837591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:31:59.837869 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:59.837648 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 07:31:59.837869 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:59.837677 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 07:31:59.837869 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:59.837702 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:32:00.837688916 +0000 UTC m=+252.362042674 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : configmap "kube-rbac-proxy" not found Sep 16 07:31:59.837869 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:31:59.837716 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:32:00.837710099 +0000 UTC m=+252.362063856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : secret "insights-runtime-extractor-tls" not found Sep 16 07:32:00.092208 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:00.092144 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:00.192774 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:00.192749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:00.844491 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:00.844451 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:00.844989 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:00.844528 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:32:00.844989 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:00.844561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:00.844989 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:00.844602 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 07:32:00.844989 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:00.844655 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 07:32:00.844989 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:00.844667 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 16 07:32:00.844989 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:00.844662 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:32:02.844647366 +0000 UTC m=+254.369001124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : secret "insights-runtime-extractor-tls" not found Sep 16 07:32:00.844989 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:00.844723 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:32:02.844704575 +0000 UTC m=+254.369058332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : configmap "kube-rbac-proxy" not found Sep 16 07:32:00.844989 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:00.844745 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs podName:e1477d2a-2a8c-4b94-b45f-d8d82ece6aab nodeName:}" failed. No retries permitted until 2026-09-16 07:34:02.844735652 +0000 UTC m=+374.369089412 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs") pod "network-metrics-daemon-99h7c" (UID: "e1477d2a-2a8c-4b94-b45f-d8d82ece6aab") : secret "metrics-daemon-secret" not found Sep 16 07:32:01.167079 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.167051 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hjlx2_92f584a7-6d57-4f6b-9f18-b49f43a95e4e/dns-node-resolver/0.log" Sep 16 07:32:01.550421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.550335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:32:01.550421 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.550392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:32:01.550598 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.550438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:32:01.550598 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:01.550490 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 07:32:01.550598 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:01.550533 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 07:32:01.550598 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:01.550549 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59548f6486-jrsg2: secret "image-registry-tls" not found Sep 16 07:32:01.550598 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:01.550560 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls podName:4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc nodeName:}" failed. No retries permitted until 2026-09-16 07:32:09.550545352 +0000 UTC m=+261.074899109 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-24djp" (UID: "4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc") : secret "cluster-monitoring-operator-tls" not found Sep 16 07:32:01.550598 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:01.550588 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls podName:ec53573a-703d-4e8e-ab3c-05f3db9253c0 nodeName:}" failed. No retries permitted until 2026-09-16 07:32:09.550577654 +0000 UTC m=+261.074931412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls") pod "image-registry-59548f6486-jrsg2" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0") : secret "image-registry-tls" not found Sep 16 07:32:01.550598 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:01.550537 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 07:32:01.550842 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:01.550623 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls podName:b8e9d14a-fed3-497f-aa3d-d7c8e783fa01 nodeName:}" failed. No retries permitted until 2026-09-16 07:32:09.550614624 +0000 UTC m=+261.074968381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-tj6dv" (UID: "b8e9d14a-fed3-497f-aa3d-d7c8e783fa01") : secret "samples-operator-tls" not found Sep 16 07:32:01.692264 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.692232 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-shw59"] Sep 16 07:32:01.695246 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.695229 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:01.697207 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.697185 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 16 07:32:01.697302 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.697219 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 16 07:32:01.697952 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.697936 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 16 07:32:01.698038 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.697940 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tzh59\"" Sep 16 07:32:01.698038 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.698000 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 16 07:32:01.702905 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.702885 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-shw59"] Sep 16 07:32:01.724305 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.724281 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:01.766964 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.766945 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-962g2_28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2/node-ca/0.log" Sep 16 07:32:01.788727 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.788703 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:01.852822 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.852720 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5-signing-cabundle\") pod \"service-ca-568869fdc5-shw59\" (UID: \"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5\") " pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:01.852822 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.852780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6gcz\" (UniqueName: \"kubernetes.io/projected/45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5-kube-api-access-b6gcz\") pod \"service-ca-568869fdc5-shw59\" (UID: \"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5\") " pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:01.853228 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.852902 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5-signing-key\") pod \"service-ca-568869fdc5-shw59\" (UID: \"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5\") " pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:01.889389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.889359 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:01.953386 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.953359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5-signing-cabundle\") pod \"service-ca-568869fdc5-shw59\" (UID: \"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5\") " pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:01.953500 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.953424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6gcz\" (UniqueName: \"kubernetes.io/projected/45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5-kube-api-access-b6gcz\") pod \"service-ca-568869fdc5-shw59\" (UID: \"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5\") " pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:01.953654 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.953634 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5-signing-key\") pod \"service-ca-568869fdc5-shw59\" (UID: \"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5\") " pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:01.953997 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.953981 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5-signing-cabundle\") pod \"service-ca-568869fdc5-shw59\" (UID: \"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5\") " pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:01.955900 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.955882 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5-signing-key\") pod \"service-ca-568869fdc5-shw59\" (UID: \"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5\") " pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:01.961825 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:01.961805 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6gcz\" (UniqueName: \"kubernetes.io/projected/45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5-kube-api-access-b6gcz\") pod \"service-ca-568869fdc5-shw59\" (UID: \"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5\") " pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:02.004751 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:02.004719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-shw59" Sep 16 07:32:02.283952 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:02.283916 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-shw59"] Sep 16 07:32:02.287962 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:32:02.287933 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45bcc6e9_fc0c_472d_b84d_ba7dc809f0b5.slice/crio-0e2dd154e24bf70045607c536fb96a0770422a1db6df0c853d1dded46ef0de5c WatchSource:0}: Error finding container 0e2dd154e24bf70045607c536fb96a0770422a1db6df0c853d1dded46ef0de5c: Status 404 returned error can't find the container with id 0e2dd154e24bf70045607c536fb96a0770422a1db6df0c853d1dded46ef0de5c Sep 16 07:32:02.695083 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:02.695040 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:02.822029 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:02.821998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-shw59" event={"ID":"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5","Type":"ContainerStarted","Data":"0e2dd154e24bf70045607c536fb96a0770422a1db6df0c853d1dded46ef0de5c"} Sep 16 07:32:02.861377 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:02.861353 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:02.861674 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:02.861431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:02.861674 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:02.861511 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 07:32:02.861674 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:02.861567 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:32:06.861553945 +0000 UTC m=+258.385907706 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : secret "insights-runtime-extractor-tls" not found Sep 16 07:32:02.861674 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:02.861511 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 07:32:02.861674 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:02.861657 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:32:06.861643536 +0000 UTC m=+258.385997296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : configmap "kube-rbac-proxy" not found Sep 16 07:32:02.967847 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:02.967758 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zrfpb_5b0aee94-ad88-4d46-93f1-01ccd630a7cd/serve-healthcheck-canary/0.log" Sep 16 07:32:04.082101 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:04.082020 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:32:04.082493 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:04.082101 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:32:04.082493 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:04.082412 2580 scope.go:117] "RemoveContainer" containerID="ae22d57d0f3c26eb1c44e5939d8584d308d994cf78fa6fca141ff737d8e51c7d" Sep 16 07:32:04.082585 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:04.082566 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-bhpfr_openshift-console-operator(84595af1-2295-41ac-abe5-c448a3116f98)\"" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" podUID="84595af1-2295-41ac-abe5-c448a3116f98" Sep 16 07:32:04.830199 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:04.830167 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-shw59" event={"ID":"45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5","Type":"ContainerStarted","Data":"c6c753392af9d0f00c22b140c4724b2f8f4f66ef49ea0db8ec8b6e2f24a1effa"} Sep 16 07:32:04.830511 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:04.830492 2580 scope.go:117] "RemoveContainer" containerID="ae22d57d0f3c26eb1c44e5939d8584d308d994cf78fa6fca141ff737d8e51c7d" Sep 16 07:32:04.830683 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:04.830661 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-bhpfr_openshift-console-operator(84595af1-2295-41ac-abe5-c448a3116f98)\"" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" podUID="84595af1-2295-41ac-abe5-c448a3116f98" Sep 16 07:32:04.847986 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:04.847946 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-shw59" podStartSLOduration=1.8693795450000001 podStartE2EDuration="3.847933058s" totalImagesPullingTime="1.978553513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:32:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:04.846618214 +0000 UTC m=+256.370971993" watchObservedRunningTime="2026-09-16 07:32:04.847933058 +0000 UTC m=+256.372286834" Sep 16 07:32:05.779949 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:05.779905 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:05.881490 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:05.881454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:05.981695 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:05.981655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:06.085864 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:06.085774 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:06.865522 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:06.865492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:06.892946 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:06.892916 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:06.893074 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:06.892982 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:06.893074 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:06.893052 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 07:32:06.893159 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:06.893074 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 07:32:06.893159 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:06.893115 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:32:14.893101725 +0000 UTC m=+266.417455497 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : secret "insights-runtime-extractor-tls" not found Sep 16 07:32:06.893159 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:06.893130 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:32:14.893122825 +0000 UTC m=+266.417476582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : configmap "kube-rbac-proxy" not found Sep 16 07:32:07.470740 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:07.470693 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:07.669887 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:07.669851 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:08.064537 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:08.064502 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:08.267654 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:08.267605 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:08.458865 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:08.458820 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:09.474596 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.474559 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:09.532748 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.532717 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:09.612055 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.612015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:32:09.612221 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.612079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:32:09.612221 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.612120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:32:09.612221 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:09.612170 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 07:32:09.612375 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:09.612241 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls podName:4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc nodeName:}" failed. No retries permitted until 2026-09-16 07:32:25.61222511 +0000 UTC m=+277.136578868 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-24djp" (UID: "4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc") : secret "cluster-monitoring-operator-tls" not found Sep 16 07:32:09.614462 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.614432 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8e9d14a-fed3-497f-aa3d-d7c8e783fa01-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-tj6dv\" (UID: \"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:32:09.614582 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.614466 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") pod \"image-registry-59548f6486-jrsg2\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:32:09.655372 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.655337 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" Sep 16 07:32:09.666666 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.666638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:09.675412 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.675388 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:32:09.864524 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.864451 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:09.923958 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.923936 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv"] Sep 16 07:32:09.951167 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:09.951141 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59548f6486-jrsg2"] Sep 16 07:32:09.954226 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:32:09.954200 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec53573a_703d_4e8e_ab3c_05f3db9253c0.slice/crio-34f23430de3f8165ab8a1c0fb467b8b49188b70faeafb397625c502944883709 WatchSource:0}: Error finding container 34f23430de3f8165ab8a1c0fb467b8b49188b70faeafb397625c502944883709: Status 404 returned error can't find the container with id 34f23430de3f8165ab8a1c0fb467b8b49188b70faeafb397625c502944883709 Sep 16 07:32:10.066211 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:10.066185 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:10.263768 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:10.263645 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:10.467151 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:10.467094 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:10.852138 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:10.852102 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" event={"ID":"ec53573a-703d-4e8e-ab3c-05f3db9253c0","Type":"ContainerStarted","Data":"724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed"} Sep 16 07:32:10.852601 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:10.852158 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" event={"ID":"ec53573a-703d-4e8e-ab3c-05f3db9253c0","Type":"ContainerStarted","Data":"34f23430de3f8165ab8a1c0fb467b8b49188b70faeafb397625c502944883709"} Sep 16 07:32:10.852601 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:10.852214 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:32:10.853652 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:10.853626 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" event={"ID":"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01","Type":"ContainerStarted","Data":"d55ca8c3ad8934d988a18597c47ac9c59a73eb0e03b0343a4e112a229451c2da"} Sep 16 07:32:10.872404 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:10.872352 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" podStartSLOduration=17.872338136 podStartE2EDuration="17.872338136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:10.871261767 +0000 UTC m=+262.395615548" watchObservedRunningTime="2026-09-16 07:32:10.872338136 +0000 UTC m=+262.396691959" Sep 16 07:32:11.469450 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:11.469401 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:11.664186 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:11.664032 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:11.858143 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:11.858107 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" event={"ID":"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01","Type":"ContainerStarted","Data":"abf1abad4b8fa7e473ed03b61509700f1470dd3189ad478c4d49b5b41add60c1"} Sep 16 07:32:11.858561 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:11.858171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" event={"ID":"b8e9d14a-fed3-497f-aa3d-d7c8e783fa01","Type":"ContainerStarted","Data":"b6ebe61987146b0fcf1a45f5f37c01a6546408f0de2676b5fb3789dfd761fb8d"} Sep 16 07:32:11.873721 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:11.873685 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-tj6dv" podStartSLOduration=17.256925465 podStartE2EDuration="18.873671085s" totalImagesPullingTime="1.61674562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:11.872967976 +0000 UTC m=+263.397321765" watchObservedRunningTime="2026-09-16 07:32:11.873671085 +0000 UTC m=+263.398024860" Sep 16 07:32:12.069689 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:12.069662 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:12.266508 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:12.266446 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:12.324126 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:12.324101 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:12.667946 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:12.667913 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:13.471181 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:13.471146 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:14.762469 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:14.762434 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:14.954076 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:14.954045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:14.954223 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:14.954102 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:14.954223 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:14.954210 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 07:32:14.954293 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:14.954258 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls podName:2c9e87ea-05bc-40ce-ad08-37a06216c67c nodeName:}" failed. No retries permitted until 2026-09-16 07:32:30.954245835 +0000 UTC m=+282.478599591 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2gjlc" (UID: "2c9e87ea-05bc-40ce-ad08-37a06216c67c") : secret "insights-runtime-extractor-tls" not found Sep 16 07:32:14.954552 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:14.954535 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c9e87ea-05bc-40ce-ad08-37a06216c67c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:15.270368 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:15.270334 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:15.866858 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:15.866824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:16.066038 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:16.066003 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:16.267414 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:16.267338 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:16.468978 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:16.468944 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:18.379526 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:18.379494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:18.447972 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:18.447941 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:20.055487 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:20.055212 2580 scope.go:117] "RemoveContainer" containerID="ae22d57d0f3c26eb1c44e5939d8584d308d994cf78fa6fca141ff737d8e51c7d" Sep 16 07:32:20.887367 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:20.887339 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:32:20.887518 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:20.887423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" event={"ID":"84595af1-2295-41ac-abe5-c448a3116f98","Type":"ContainerStarted","Data":"37d8e45cc80d4ad2c2c136a68c131a1f45ca1167d4f4a9e64274785d548886ae"} Sep 16 07:32:20.887721 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:20.887702 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:32:20.892197 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:20.892177 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" Sep 16 07:32:20.903561 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:20.903521 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-bhpfr" podStartSLOduration=25.819590278 podStartE2EDuration="27.903509038s" totalImagesPullingTime="2.08391876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:20.903167857 +0000 UTC m=+272.427521637" watchObservedRunningTime="2026-09-16 07:32:20.903509038 +0000 UTC m=+272.427862813" Sep 16 07:32:20.903957 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:20.903938 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:21.002417 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:21.002389 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:21.067918 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:21.067891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:21.202208 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:21.202148 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:21.584648 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:21.584566 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:21.639747 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:21.639719 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:21.986113 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:21.986082 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:22.085762 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.085732 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:22.369173 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.369146 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-tn5cw"] Sep 16 07:32:22.372388 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.372370 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-tn5cw" Sep 16 07:32:22.375200 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.375179 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 16 07:32:22.375369 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.375350 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 16 07:32:22.375440 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.375410 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bl8jd\"" Sep 16 07:32:22.386599 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.386580 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-tn5cw"] Sep 16 07:32:22.407172 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.407146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9nn\" (UniqueName: \"kubernetes.io/projected/cdb9ee11-ed4b-4669-bf34-83514dc6adb3-kube-api-access-nw9nn\") pod \"downloads-76f99584f4-tn5cw\" (UID: \"cdb9ee11-ed4b-4669-bf34-83514dc6adb3\") " pod="openshift-console/downloads-76f99584f4-tn5cw" Sep 16 07:32:22.407974 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.407958 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59548f6486-jrsg2"] Sep 16 07:32:22.465988 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.465960 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5565947dd6-vvjtz"] Sep 16 07:32:22.471846 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.471825 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.487303 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.487284 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5565947dd6-vvjtz"] Sep 16 07:32:22.504159 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.504133 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:22.508305 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.508288 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-bound-sa-token\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.508397 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.508327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-registry-certificates\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.508453 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.508393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-trusted-ca\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.508493 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.508477 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-ca-trust-extracted\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.508527 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.508504 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-image-registry-private-configuration\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.508562 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.508526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltpfq\" (UniqueName: \"kubernetes.io/projected/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-kube-api-access-ltpfq\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.508613 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.508564 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9nn\" (UniqueName: \"kubernetes.io/projected/cdb9ee11-ed4b-4669-bf34-83514dc6adb3-kube-api-access-nw9nn\") pod \"downloads-76f99584f4-tn5cw\" (UID: \"cdb9ee11-ed4b-4669-bf34-83514dc6adb3\") " pod="openshift-console/downloads-76f99584f4-tn5cw" Sep 16 07:32:22.508665 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.508615 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-registry-tls\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.508717 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.508664 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-installation-pull-secrets\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.523682 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.523657 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9nn\" (UniqueName: \"kubernetes.io/projected/cdb9ee11-ed4b-4669-bf34-83514dc6adb3-kube-api-access-nw9nn\") pod \"downloads-76f99584f4-tn5cw\" (UID: \"cdb9ee11-ed4b-4669-bf34-83514dc6adb3\") " pod="openshift-console/downloads-76f99584f4-tn5cw" Sep 16 07:32:22.609425 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.609404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-registry-tls\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.609535 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.609455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-installation-pull-secrets\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.609535 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.609485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-bound-sa-token\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.609535 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.609529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-registry-certificates\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.609710 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.609552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-trusted-ca\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.609757 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.609726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-ca-trust-extracted\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.609848 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.609765 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-image-registry-private-configuration\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.609848 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.609828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltpfq\" (UniqueName: \"kubernetes.io/projected/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-kube-api-access-ltpfq\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.610086 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.610065 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-ca-trust-extracted\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.610469 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.610444 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-registry-certificates\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.610551 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.610522 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-trusted-ca\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.611808 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.611773 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-registry-tls\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.611884 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.611840 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-installation-pull-secrets\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.612168 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.612147 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-image-registry-private-configuration\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.616484 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.616458 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:22.624188 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.624137 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-bound-sa-token\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.624588 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.624569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltpfq\" (UniqueName: \"kubernetes.io/projected/1f53e1b9-b774-4fb4-89c8-a4bcf604e295-kube-api-access-ltpfq\") pod \"image-registry-5565947dd6-vvjtz\" (UID: \"1f53e1b9-b774-4fb4-89c8-a4bcf604e295\") " pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.680704 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.680683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-tn5cw" Sep 16 07:32:22.779895 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.779858 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:22.809682 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.809648 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:22.950394 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:22.949378 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-tn5cw"] Sep 16 07:32:23.009029 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:23.008999 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:23.058951 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:23.058930 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5565947dd6-vvjtz"] Sep 16 07:32:23.059625 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:32:23.059604 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f53e1b9_b774_4fb4_89c8_a4bcf604e295.slice/crio-29fc572274a1f3d1f342f09337ba5a411c27cd20caac1114f201eee64b759110 WatchSource:0}: Error finding container 29fc572274a1f3d1f342f09337ba5a411c27cd20caac1114f201eee64b759110: Status 404 returned error can't find the container with id 29fc572274a1f3d1f342f09337ba5a411c27cd20caac1114f201eee64b759110 Sep 16 07:32:23.108310 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:23.108284 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:23.898128 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:23.898089 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" event={"ID":"1f53e1b9-b774-4fb4-89c8-a4bcf604e295","Type":"ContainerStarted","Data":"6ea202eafbce1a4c97c7c2aafb8e5bfa855d37aa0f158f6624d2a57675c0cdf8"} Sep 16 07:32:23.898333 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:23.898153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" event={"ID":"1f53e1b9-b774-4fb4-89c8-a4bcf604e295","Type":"ContainerStarted","Data":"29fc572274a1f3d1f342f09337ba5a411c27cd20caac1114f201eee64b759110"} Sep 16 07:32:23.898333 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:23.898214 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:23.899426 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:23.899401 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-tn5cw" event={"ID":"cdb9ee11-ed4b-4669-bf34-83514dc6adb3","Type":"ContainerStarted","Data":"263273704db556845b7e79279b7eafb3a8e22cb0994ae6a283db66d59566e8ab"} Sep 16 07:32:23.918656 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:23.918606 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" podStartSLOduration=1.918589403 podStartE2EDuration="1.918589403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:32:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:23.918184687 +0000 UTC m=+275.442538490" watchObservedRunningTime="2026-09-16 07:32:23.918589403 +0000 UTC m=+275.442943184" Sep 16 07:32:25.636013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:25.635971 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:32:25.638746 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:25.638713 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-24djp\" (UID: \"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:32:25.868337 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:25.868304 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" Sep 16 07:32:26.143622 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:26.143593 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp"] Sep 16 07:32:26.146984 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:32:26.146952 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2e49ce_6d4b_4ae6_a83c_84a8acc85bdc.slice/crio-f73c84ff3f033198eda1fbb63183819cfcd809544f3ec1adf5ac43e0e507438c WatchSource:0}: Error finding container f73c84ff3f033198eda1fbb63183819cfcd809544f3ec1adf5ac43e0e507438c: Status 404 returned error can't find the container with id f73c84ff3f033198eda1fbb63183819cfcd809544f3ec1adf5ac43e0e507438c Sep 16 07:32:26.912298 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:26.912204 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" event={"ID":"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc","Type":"ContainerStarted","Data":"f73c84ff3f033198eda1fbb63183819cfcd809544f3ec1adf5ac43e0e507438c"} Sep 16 07:32:28.372150 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:28.372115 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:28.472868 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:28.472827 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:28.556506 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:28.556472 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:28.812994 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:28.812889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:28.922087 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:28.922044 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" event={"ID":"4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc","Type":"ContainerStarted","Data":"af13dbf2c644c3831ae5d23f0e66ac5bf9ffcd2225b4820af70ab236167c3401"} Sep 16 07:32:28.938833 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:28.938771 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-24djp" podStartSLOduration=34.034607303 podStartE2EDuration="35.938757523s" totalImagesPullingTime="1.90415022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:28.938245264 +0000 UTC m=+280.462599047" watchObservedRunningTime="2026-09-16 07:32:28.938757523 +0000 UTC m=+280.463111301" Sep 16 07:32:29.577076 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:29.577028 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vmll6" podUID="ddf610bc-54f4-45e1-9c24-dec4ce0048b6" Sep 16 07:32:29.925685 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:29.925652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmll6" Sep 16 07:32:30.978937 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:30.978897 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:30.981293 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:30.981261 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c9e87ea-05bc-40ce-ad08-37a06216c67c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2gjlc\" (UID: \"2c9e87ea-05bc-40ce-ad08-37a06216c67c\") " pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:31.157845 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:31.157808 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2gjlc" Sep 16 07:32:31.434320 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:31.434295 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2gjlc"] Sep 16 07:32:31.436861 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:32:31.436829 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c9e87ea_05bc_40ce_ad08_37a06216c67c.slice/crio-a73626613a01a5450c81e689e37b451b69a6136d0c792a160bf022e2a0d27243 WatchSource:0}: Error finding container a73626613a01a5450c81e689e37b451b69a6136d0c792a160bf022e2a0d27243: Status 404 returned error can't find the container with id a73626613a01a5450c81e689e37b451b69a6136d0c792a160bf022e2a0d27243 Sep 16 07:32:31.934277 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:31.934242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2gjlc" event={"ID":"2c9e87ea-05bc-40ce-ad08-37a06216c67c","Type":"ContainerStarted","Data":"f38c9022d56668dd3d2a20e1e9be2c2678f3fd5999e3fe031e92d761fcea5e7d"} Sep 16 07:32:31.934435 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:31.934311 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2gjlc" event={"ID":"2c9e87ea-05bc-40ce-ad08-37a06216c67c","Type":"ContainerStarted","Data":"a73626613a01a5450c81e689e37b451b69a6136d0c792a160bf022e2a0d27243"} Sep 16 07:32:32.415152 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:32.415118 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:32:32.447808 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:32.447757 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:32.516056 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:32.516016 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:32.601125 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:32.601089 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:32.661864 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:32.661824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:32.893966 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:32.893925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:32:32.896389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:32.896362 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ddf610bc-54f4-45e1-9c24-dec4ce0048b6-metrics-tls\") pod \"dns-default-vmll6\" (UID: \"ddf610bc-54f4-45e1-9c24-dec4ce0048b6\") " pod="openshift-dns/dns-default-vmll6" Sep 16 07:32:32.928734 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:32.928707 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8grdh\"" Sep 16 07:32:32.936775 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:32.936755 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmll6" Sep 16 07:32:33.087388 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:33.087353 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:34.944360 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:34.944311 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:35.005429 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:35.005394 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:35.338494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:35.338416 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:35.521994 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:35.521959 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:35.584438 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:35.584399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:35.932476 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:35.932433 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:35.992149 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:35.992114 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:36.231201 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:36.231122 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:36.291432 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:36.291399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:37.698305 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.698268 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:37.906458 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.906423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:37.923205 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.922806 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62"] Sep 16 07:32:37.927041 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.927014 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:37.929824 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.929485 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 16 07:32:37.929824 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.929509 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 16 07:32:37.929824 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.929635 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-n5wn6\"" Sep 16 07:32:37.930696 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.930677 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 16 07:32:37.939102 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.939082 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62"] Sep 16 07:32:37.945652 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.945595 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gtqlp"] Sep 16 07:32:37.949177 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.949133 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:37.952273 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.951433 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 16 07:32:37.952273 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.951660 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 16 07:32:37.952273 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.951951 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-d7lb2\"" Sep 16 07:32:37.952273 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.952116 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 16 07:32:37.992434 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:37.992406 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:38.034367 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034336 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqggj\" (UniqueName: \"kubernetes.io/projected/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-kube-api-access-gqggj\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.034507 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034379 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/666114d6-52da-4dab-b0b6-2ff0c292ab30-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.034507 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-textfile\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.034507 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034442 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/666114d6-52da-4dab-b0b6-2ff0c292ab30-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.034507 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034462 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-metrics-client-ca\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.034507 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034494 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-sys\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.034738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-root\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.034738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034547 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.034738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034566 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-tls\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.034738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-wtmp\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.034738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8skls\" (UniqueName: \"kubernetes.io/projected/666114d6-52da-4dab-b0b6-2ff0c292ab30-kube-api-access-8skls\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.034738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/666114d6-52da-4dab-b0b6-2ff0c292ab30-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.034738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.034716 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-accelerators-collector-config\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.096173 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.096140 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:38.136124 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-metrics-client-ca\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136263 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-sys\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136263 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-root\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136263 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136263 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136226 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-sys\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136263 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-tls\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136298 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-wtmp\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136309 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-root\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8skls\" (UniqueName: \"kubernetes.io/projected/666114d6-52da-4dab-b0b6-2ff0c292ab30-kube-api-access-8skls\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.136513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/666114d6-52da-4dab-b0b6-2ff0c292ab30-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.136513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136433 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-accelerators-collector-config\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136456 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-wtmp\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqggj\" (UniqueName: \"kubernetes.io/projected/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-kube-api-access-gqggj\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/666114d6-52da-4dab-b0b6-2ff0c292ab30-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.136916 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-textfile\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.136916 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/666114d6-52da-4dab-b0b6-2ff0c292ab30-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.136916 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.136894 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-metrics-client-ca\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.137068 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.137044 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-accelerators-collector-config\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.137156 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:38.137134 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 16 07:32:38.137218 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:38.137199 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-tls podName:a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25 nodeName:}" failed. No retries permitted until 2026-09-16 07:32:38.637180116 +0000 UTC m=+290.161533887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-tls") pod "node-exporter-gtqlp" (UID: "a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25") : secret "node-exporter-tls" not found Sep 16 07:32:38.137431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.137409 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/666114d6-52da-4dab-b0b6-2ff0c292ab30-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.137735 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.137692 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-textfile\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.140104 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.140049 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/666114d6-52da-4dab-b0b6-2ff0c292ab30-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.140386 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.140344 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/666114d6-52da-4dab-b0b6-2ff0c292ab30-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.140516 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.140487 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.156382 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.156360 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8skls\" (UniqueName: \"kubernetes.io/projected/666114d6-52da-4dab-b0b6-2ff0c292ab30-kube-api-access-8skls\") pod \"openshift-state-metrics-67dcf8c979-mxs62\" (UID: \"666114d6-52da-4dab-b0b6-2ff0c292ab30\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.158943 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.158924 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqggj\" (UniqueName: \"kubernetes.io/projected/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-kube-api-access-gqggj\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.219583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.219430 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:38.241398 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.241346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" Sep 16 07:32:38.321075 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.321008 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:38.598110 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.597994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:38.640969 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.640933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-tls\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.643463 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.643409 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25-node-exporter-tls\") pod \"node-exporter-gtqlp\" (UID: \"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25\") " pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.697834 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.697768 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:38.863217 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.863183 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gtqlp" Sep 16 07:32:38.964159 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:38.964100 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:39.071978 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.071946 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 16 07:32:39.075958 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.075929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:39.076101 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.075966 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.079084 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.079057 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 16 07:32:39.079203 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.079060 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 16 07:32:39.079387 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.079368 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 16 07:32:39.079453 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.079398 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 16 07:32:39.079640 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.079622 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 16 07:32:39.080308 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.079965 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 16 07:32:39.080308 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.080101 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 16 07:32:39.080308 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.080172 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r76rv\"" Sep 16 07:32:39.087821 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.087704 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 16 07:32:39.094241 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.094215 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 16 07:32:39.101281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.101208 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145347 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145419 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-out\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145491 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l7cf\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-kube-api-access-2l7cf\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145561 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145589 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-web-config\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145659 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145686 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145748 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.146433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.145779 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-volume\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.147386 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:32:39.147350 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1aaf3ec_6b99_4e21_b908_e8c8f7a14d25.slice/crio-11134334caf3af91f829bc3e99c8d3635d8425787f2eca7d66c3000a8c7fe79e WatchSource:0}: Error finding container 11134334caf3af91f829bc3e99c8d3635d8425787f2eca7d66c3000a8c7fe79e: Status 404 returned error can't find the container with id 11134334caf3af91f829bc3e99c8d3635d8425787f2eca7d66c3000a8c7fe79e Sep 16 07:32:39.246949 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.246876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-volume\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.246949 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.246930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247159 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.246983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-out\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247159 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247159 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247055 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247159 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247159 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247105 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l7cf\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-kube-api-access-2l7cf\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247159 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247135 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247490 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-web-config\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247490 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247490 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247490 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.247490 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.247328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.248463 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.248440 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.248659 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.248630 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.249700 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:39.248845 2580 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 16 07:32:39.249700 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.248894 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.249700 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:39.248909 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-main-tls podName:19103d9b-c62c-40c1-adfe-5ffcdad81a10 nodeName:}" failed. No retries permitted until 2026-09-16 07:32:39.748890813 +0000 UTC m=+291.273244588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10") : secret "alertmanager-main-tls" not found Sep 16 07:32:39.251299 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.251255 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-out\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.253069 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.253026 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-web-config\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.253583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.253549 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.253654 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.253577 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.254014 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.253990 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-volume\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.254472 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.254448 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.254550 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.254464 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.254857 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.254835 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.256407 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.256366 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l7cf\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-kube-api-access-2l7cf\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.416120 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.416042 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62"] Sep 16 07:32:39.420626 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:32:39.420599 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod666114d6_52da_4dab_b0b6_2ff0c292ab30.slice/crio-e598ce1ca16452a95680b66c6c003d0bd4f3cc5fcf0440f0ea92e2e99aeae230 WatchSource:0}: Error finding container e598ce1ca16452a95680b66c6c003d0bd4f3cc5fcf0440f0ea92e2e99aeae230: Status 404 returned error can't find the container with id e598ce1ca16452a95680b66c6c003d0bd4f3cc5fcf0440f0ea92e2e99aeae230 Sep 16 07:32:39.424553 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.424523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:39.434513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.434190 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vmll6"] Sep 16 07:32:39.439325 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:32:39.439287 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf610bc_54f4_45e1_9c24_dec4ce0048b6.slice/crio-994a3257d4e65f96130c451791894ac167e9858da437e72dd04e289084ec79af WatchSource:0}: Error finding container 994a3257d4e65f96130c451791894ac167e9858da437e72dd04e289084ec79af: Status 404 returned error can't find the container with id 994a3257d4e65f96130c451791894ac167e9858da437e72dd04e289084ec79af Sep 16 07:32:39.513736 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.513699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:39.708040 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.707905 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:39.753322 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.752719 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.756089 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.756048 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.982009 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.981827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2gjlc" event={"ID":"2c9e87ea-05bc-40ce-ad08-37a06216c67c","Type":"ContainerStarted","Data":"ec00108bcb249c67e3104200a2b60784f6542a89a7965c78f5d777e2b28c0b30"} Sep 16 07:32:39.984609 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.984544 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gtqlp" event={"ID":"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25","Type":"ContainerStarted","Data":"11134334caf3af91f829bc3e99c8d3635d8425787f2eca7d66c3000a8c7fe79e"} Sep 16 07:32:39.986830 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.986803 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmll6" event={"ID":"ddf610bc-54f4-45e1-9c24-dec4ce0048b6","Type":"ContainerStarted","Data":"994a3257d4e65f96130c451791894ac167e9858da437e72dd04e289084ec79af"} Sep 16 07:32:39.987804 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.987721 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:32:39.990397 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.990261 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-tn5cw" event={"ID":"cdb9ee11-ed4b-4669-bf34-83514dc6adb3","Type":"ContainerStarted","Data":"4a2d5ea1e388de0f4846079bf6513e220912668578a9169e8d99f8c2a0486b09"} Sep 16 07:32:39.993210 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.990907 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-tn5cw" Sep 16 07:32:39.993667 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.993621 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" event={"ID":"666114d6-52da-4dab-b0b6-2ff0c292ab30","Type":"ContainerStarted","Data":"7bb1c1426bc55c88e5bc0de59bf536ace087fadd872412fdd514d4701d269f8c"} Sep 16 07:32:39.993755 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.993671 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" event={"ID":"666114d6-52da-4dab-b0b6-2ff0c292ab30","Type":"ContainerStarted","Data":"9538ad3aff2d04b7df26975f48e99cc8954543adf29bea26b70a911d02605263"} Sep 16 07:32:39.993755 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.993702 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" event={"ID":"666114d6-52da-4dab-b0b6-2ff0c292ab30","Type":"ContainerStarted","Data":"e598ce1ca16452a95680b66c6c003d0bd4f3cc5fcf0440f0ea92e2e99aeae230"} Sep 16 07:32:39.997453 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:39.997412 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-tn5cw" Sep 16 07:32:40.008101 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:40.008053 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-tn5cw" podStartSLOduration=1.6130435410000001 podStartE2EDuration="18.008033806s" totalImagesPullingTime="16.394990265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:40.006393641 +0000 UTC m=+291.530747422" watchObservedRunningTime="2026-09-16 07:32:40.008033806 +0000 UTC m=+291.532387577" Sep 16 07:32:40.339170 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:40.339135 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 16 07:32:40.500582 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:40.500494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:40.599810 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:40.599656 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:40.704287 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:40.704141 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:40.805742 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:32:40.805704 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19103d9b_c62c_40c1_adfe_5ffcdad81a10.slice/crio-43f778e02b84e85d7cb91975890a06bfd388bc908cb159c444d6f512f2515b23 WatchSource:0}: Error finding container 43f778e02b84e85d7cb91975890a06bfd388bc908cb159c444d6f512f2515b23: Status 404 returned error can't find the container with id 43f778e02b84e85d7cb91975890a06bfd388bc908cb159c444d6f512f2515b23 Sep 16 07:32:40.899945 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:40.899764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:41.003325 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:41.003198 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerStarted","Data":"43f778e02b84e85d7cb91975890a06bfd388bc908cb159c444d6f512f2515b23"} Sep 16 07:32:41.018275 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:41.017636 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:41.103019 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:41.102975 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:41.814599 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:41.814561 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:42.722411 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:42.722375 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:42.825458 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:42.825368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:42.975678 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:42.975599 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:43.823583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:43.823504 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:43.924245 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:43.924166 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:44.017075 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:44.017048 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gtqlp" event={"ID":"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25","Type":"ContainerStarted","Data":"c140ff140794fcb4a9f961ac766114e128be0c3180a2ecfbe2ab0065593618d2"} Sep 16 07:32:44.061250 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:44.061226 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:44.161466 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:44.161434 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:44.248481 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:44.248447 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:44.908451 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:44.908424 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5565947dd6-vvjtz" Sep 16 07:32:45.031848 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.031813 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" event={"ID":"666114d6-52da-4dab-b0b6-2ff0c292ab30","Type":"ContainerStarted","Data":"2a3b36b660c42de0203d24dacac24690173cc315a5236f589a2427702a23f113"} Sep 16 07:32:45.033980 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.033952 2580 generic.go:356] "Generic (PLEG): container finished" podID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerID="444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06" exitCode=0 Sep 16 07:32:45.034133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.034042 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerDied","Data":"444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06"} Sep 16 07:32:45.036945 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.036917 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2gjlc" event={"ID":"2c9e87ea-05bc-40ce-ad08-37a06216c67c","Type":"ContainerStarted","Data":"a9025b3236c64c74cb0aa02f975c9a0403aea6f5df322e0578263af228f973e1"} Sep 16 07:32:45.038901 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.038872 2580 generic.go:356] "Generic (PLEG): container finished" podID="a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25" containerID="c140ff140794fcb4a9f961ac766114e128be0c3180a2ecfbe2ab0065593618d2" exitCode=0 Sep 16 07:32:45.038996 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.038950 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gtqlp" event={"ID":"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25","Type":"ContainerDied","Data":"c140ff140794fcb4a9f961ac766114e128be0c3180a2ecfbe2ab0065593618d2"} Sep 16 07:32:45.041167 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.041136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmll6" event={"ID":"ddf610bc-54f4-45e1-9c24-dec4ce0048b6","Type":"ContainerStarted","Data":"1554e12fc55c056653abd05ad5c28ff90fc3d158b120f59732bbc204d6801a5e"} Sep 16 07:32:45.041262 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.041193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmll6" event={"ID":"ddf610bc-54f4-45e1-9c24-dec4ce0048b6","Type":"ContainerStarted","Data":"f20f8b3d8f112c5467a02ae45a53235cb61a4cbea444f874efcf9dc89fd31933"} Sep 16 07:32:45.041327 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.041300 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vmll6" Sep 16 07:32:45.055524 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:45.055491 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-99h7c" podUID="e1477d2a-2a8c-4b94-b45f-d8d82ece6aab" Sep 16 07:32:45.095116 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.095077 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-mxs62" podStartSLOduration=3.810251581 podStartE2EDuration="8.095062436s" totalImagesPullingTime="4.284810855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:32:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:45.087132234 +0000 UTC m=+296.611486024" watchObservedRunningTime="2026-09-16 07:32:45.095062436 +0000 UTC m=+296.619416214" Sep 16 07:32:45.120132 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.120089 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vmll6" podStartSLOduration=258.719968545 podStartE2EDuration="4m23.120072324s" totalImagesPullingTime="4.400103779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:45.118721597 +0000 UTC m=+296.643075378" watchObservedRunningTime="2026-09-16 07:32:45.120072324 +0000 UTC m=+296.644426104" Sep 16 07:32:45.131847 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.131820 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:45.145888 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.145821 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2gjlc" podStartSLOduration=33.879774308 podStartE2EDuration="46.14580265s" totalImagesPullingTime="12.266028342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 07:31:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:45.144697655 +0000 UTC m=+296.669051432" watchObservedRunningTime="2026-09-16 07:32:45.14580265 +0000 UTC m=+296.670156427" Sep 16 07:32:45.224226 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.224144 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:45.299901 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:45.299806 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:46.049964 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:46.049930 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gtqlp" event={"ID":"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25","Type":"ContainerStarted","Data":"37af9cfcf620399c37bdfbc6636a401e1ede1db9627f8e326d72739a468ce2f1"} Sep 16 07:32:46.050430 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:46.049990 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gtqlp" event={"ID":"a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25","Type":"ContainerStarted","Data":"7857ef15380ea355ce9a2170c16b01ddd7bf4bac507b66ad74763ee3dd3f9f0c"} Sep 16 07:32:46.071223 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:46.071166 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gtqlp" podStartSLOduration=4.291605811 podStartE2EDuration="9.071149065s" totalImagesPullingTime="4.691416714s" totalInitContainerRuntime="88.12654ms" isStatefulPod=true podCreationTimestamp="2026-09-16 07:32:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:46.069236336 +0000 UTC m=+297.593590116" watchObservedRunningTime="2026-09-16 07:32:46.071149065 +0000 UTC m=+297.595502845" Sep 16 07:32:46.821618 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:46.821582 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:46.921389 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:46.921351 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:47.426230 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.426196 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" podUID="ec53573a-703d-4e8e-ab3c-05f3db9253c0" containerName="registry" containerID="cri-o://724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed" gracePeriod=30 Sep 16 07:32:47.721388 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.721336 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:32:47.833763 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.833732 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-image-registry-private-configuration\") pod \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " Sep 16 07:32:47.833889 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.833797 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-bound-sa-token\") pod \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " Sep 16 07:32:47.833889 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.833856 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-installation-pull-secrets\") pod \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " Sep 16 07:32:47.834007 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.833890 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5klq\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-kube-api-access-t5klq\") pod \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " Sep 16 07:32:47.834076 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.834020 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") pod \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " Sep 16 07:32:47.834131 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.834082 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-certificates\") pod \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " Sep 16 07:32:47.834131 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.834116 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-trusted-ca\") pod \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " Sep 16 07:32:47.834230 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.834144 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec53573a-703d-4e8e-ab3c-05f3db9253c0-ca-trust-extracted\") pod \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\" (UID: \"ec53573a-703d-4e8e-ab3c-05f3db9253c0\") " Sep 16 07:32:47.834973 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.834925 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ec53573a-703d-4e8e-ab3c-05f3db9253c0" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 07:32:47.835194 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.835172 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ec53573a-703d-4e8e-ab3c-05f3db9253c0" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 07:32:47.836474 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.836445 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-kube-api-access-t5klq" (OuterVolumeSpecName: "kube-api-access-t5klq") pod "ec53573a-703d-4e8e-ab3c-05f3db9253c0" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0"). InnerVolumeSpecName "kube-api-access-t5klq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:32:47.836587 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.836555 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ec53573a-703d-4e8e-ab3c-05f3db9253c0" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:32:47.836587 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.836576 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ec53573a-703d-4e8e-ab3c-05f3db9253c0" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:32:47.836756 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.836705 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ec53573a-703d-4e8e-ab3c-05f3db9253c0" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:32:47.837620 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.837588 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ec53573a-703d-4e8e-ab3c-05f3db9253c0" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:32:47.846982 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.846958 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec53573a-703d-4e8e-ab3c-05f3db9253c0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ec53573a-703d-4e8e-ab3c-05f3db9253c0" (UID: "ec53573a-703d-4e8e-ab3c-05f3db9253c0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:32:47.935743 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.935677 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-image-registry-private-configuration\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:32:47.935743 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.935707 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-bound-sa-token\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:32:47.935743 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.935722 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec53573a-703d-4e8e-ab3c-05f3db9253c0-installation-pull-secrets\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:32:47.935743 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.935740 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5klq\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-kube-api-access-t5klq\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:32:47.935997 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.935750 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:32:47.935997 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.935765 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-registry-certificates\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:32:47.935997 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.935779 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec53573a-703d-4e8e-ab3c-05f3db9253c0-trusted-ca\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:32:47.935997 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:47.935800 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec53573a-703d-4e8e-ab3c-05f3db9253c0-ca-trust-extracted\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:32:48.060600 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.060565 2580 generic.go:356] "Generic (PLEG): container finished" podID="ec53573a-703d-4e8e-ab3c-05f3db9253c0" containerID="724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed" exitCode=0 Sep 16 07:32:48.060726 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.060619 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" Sep 16 07:32:48.060726 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.060660 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" event={"ID":"ec53573a-703d-4e8e-ab3c-05f3db9253c0","Type":"ContainerDied","Data":"724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed"} Sep 16 07:32:48.060726 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.060711 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59548f6486-jrsg2" event={"ID":"ec53573a-703d-4e8e-ab3c-05f3db9253c0","Type":"ContainerDied","Data":"34f23430de3f8165ab8a1c0fb467b8b49188b70faeafb397625c502944883709"} Sep 16 07:32:48.060902 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.060732 2580 scope.go:117] "RemoveContainer" containerID="724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed" Sep 16 07:32:48.064777 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.064748 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerStarted","Data":"e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62"} Sep 16 07:32:48.064903 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.064840 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerStarted","Data":"da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4"} Sep 16 07:32:48.064903 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.064889 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerStarted","Data":"e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909"} Sep 16 07:32:48.065013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.064926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerStarted","Data":"68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506"} Sep 16 07:32:48.065013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.064967 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerStarted","Data":"54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4"} Sep 16 07:32:48.073284 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.073258 2580 scope.go:117] "RemoveContainer" containerID="724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed" Sep 16 07:32:48.073612 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:32:48.073580 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed\": container with ID starting with 724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed not found: ID does not exist" containerID="724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed" Sep 16 07:32:48.073710 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.073639 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed"} err="failed to get container status \"724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed\": rpc error: code = NotFound desc = could not find container \"724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed\": container with ID starting with 724538e153bd768aa6bcd6680dff582c73195a887828778c03fdbd09fb9debed not found: ID does not exist" Sep 16 07:32:48.083415 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.083395 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59548f6486-jrsg2"] Sep 16 07:32:48.088186 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.088162 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59548f6486-jrsg2"] Sep 16 07:32:48.379621 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.379591 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:48.482363 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.482318 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:48.823142 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.823121 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:32:48.823932 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.823911 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:32:48.834544 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:48.834522 2580 kubelet.go:1648] "Image garbage collection succeeded" Sep 16 07:32:49.062689 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:49.062555 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec53573a-703d-4e8e-ab3c-05f3db9253c0" path="/var/lib/kubelet/pods/ec53573a-703d-4e8e-ab3c-05f3db9253c0/volumes" Sep 16 07:32:50.111030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:50.074380 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerStarted","Data":"d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c"} Sep 16 07:32:50.111030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:50.101177 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.289240773 podStartE2EDuration="11.101162418s" totalImagesPullingTime="6.791201234s" totalInitContainerRuntime="20.720411ms" isStatefulPod=false podCreationTimestamp="2026-09-16 07:32:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:32:50.099395384 +0000 UTC m=+301.623749162" watchObservedRunningTime="2026-09-16 07:32:50.101162418 +0000 UTC m=+301.625516196" Sep 16 07:32:54.023822 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:54.023763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:54.127618 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:54.127586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:54.188065 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:54.188033 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:54.532963 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:54.532902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:54.593191 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:54.593162 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:55.053077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:55.053050 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vmll6" Sep 16 07:32:57.115318 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:57.115293 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:57.174201 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:57.174172 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:32:58.054821 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:32:58.054772 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:33:03.381590 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:03.381556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:03.437382 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:03.437357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:03.954511 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:03.954413 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:04.236182 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:04.236065 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:04.295239 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:04.295214 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:18.388419 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:18.388381 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:18.459909 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:18.459880 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:46.196055 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:46.195977 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:46.258263 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:46.258231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:57.181171 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:57.181138 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:57.783793 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:57.783739 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:33:58.244995 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:58.244962 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 16 07:33:58.245357 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:58.245228 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="alertmanager" containerID="cri-o://54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4" gracePeriod=120 Sep 16 07:33:58.245357 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:58.245298 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy-metric" containerID="cri-o://e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62" gracePeriod=120 Sep 16 07:33:58.245357 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:58.245345 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy-web" containerID="cri-o://e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909" gracePeriod=120 Sep 16 07:33:58.245532 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:58.245356 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy" containerID="cri-o://da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4" gracePeriod=120 Sep 16 07:33:58.245532 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:58.245420 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="config-reloader" containerID="cri-o://68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506" gracePeriod=120 Sep 16 07:33:58.245532 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:58.245341 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="prom-label-proxy" containerID="cri-o://d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c" gracePeriod=120 Sep 16 07:33:59.326306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326270 2580 generic.go:356] "Generic (PLEG): container finished" podID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerID="d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c" exitCode=0 Sep 16 07:33:59.326306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326298 2580 generic.go:356] "Generic (PLEG): container finished" podID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerID="e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62" exitCode=0 Sep 16 07:33:59.326306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326306 2580 generic.go:356] "Generic (PLEG): container finished" podID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerID="da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4" exitCode=0 Sep 16 07:33:59.326306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326312 2580 generic.go:356] "Generic (PLEG): container finished" podID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerID="68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506" exitCode=0 Sep 16 07:33:59.326306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326318 2580 generic.go:356] "Generic (PLEG): container finished" podID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerID="54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4" exitCode=0 Sep 16 07:33:59.326800 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerDied","Data":"d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c"} Sep 16 07:33:59.326800 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326424 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerDied","Data":"e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62"} Sep 16 07:33:59.326800 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326451 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerDied","Data":"da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4"} Sep 16 07:33:59.326800 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326475 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerDied","Data":"68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506"} Sep 16 07:33:59.326800 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.326498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerDied","Data":"54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4"} Sep 16 07:33:59.481559 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.481538 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:33:59.642451 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642427 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l7cf\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-kube-api-access-2l7cf\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642461 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-cluster-tls-config\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642489 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-trusted-ca-bundle\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642543 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-volume\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642578 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-out\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642625 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642649 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-web-config\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642679 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-web\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642703 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-tls-assets\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642731 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-main-db\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.642827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642819 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.643127 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642844 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-metrics-client-ca\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.643127 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642872 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-main-tls\") pod \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\" (UID: \"19103d9b-c62c-40c1-adfe-5ffcdad81a10\") " Sep 16 07:33:59.643127 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.642946 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 07:33:59.643273 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.643131 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.643936 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.643908 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 07:33:59.645871 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.645365 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-out" (OuterVolumeSpecName: "config-out") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:33:59.645871 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.645644 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:33:59.645871 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.645740 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:33:59.645871 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.645808 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-volume" (OuterVolumeSpecName: "config-volume") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:33:59.645871 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.645829 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:33:59.645871 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.645842 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-kube-api-access-2l7cf" (OuterVolumeSpecName: "kube-api-access-2l7cf") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "kube-api-access-2l7cf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:33:59.646231 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.646087 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:33:59.646231 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.646124 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:33:59.647428 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.647400 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:33:59.649250 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.649192 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:33:59.656013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.655991 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-web-config" (OuterVolumeSpecName: "web-config") pod "19103d9b-c62c-40c1-adfe-5ffcdad81a10" (UID: "19103d9b-c62c-40c1-adfe-5ffcdad81a10"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:33:59.744102 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744076 2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-volume\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744102 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744098 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-config-out\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744113 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744126 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-web-config\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744140 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744153 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-tls-assets\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744166 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19103d9b-c62c-40c1-adfe-5ffcdad81a10-alertmanager-main-db\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744181 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744194 2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19103d9b-c62c-40c1-adfe-5ffcdad81a10-metrics-client-ca\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744207 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-secret-alertmanager-main-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744219 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l7cf\" (UniqueName: \"kubernetes.io/projected/19103d9b-c62c-40c1-adfe-5ffcdad81a10-kube-api-access-2l7cf\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:33:59.744254 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:33:59.744233 2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19103d9b-c62c-40c1-adfe-5ffcdad81a10-cluster-tls-config\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:34:00.333652 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.333618 2580 generic.go:356] "Generic (PLEG): container finished" podID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerID="e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909" exitCode=0 Sep 16 07:34:00.334041 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.333660 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerDied","Data":"e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909"} Sep 16 07:34:00.334041 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.333696 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.334041 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.333737 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19103d9b-c62c-40c1-adfe-5ffcdad81a10","Type":"ContainerDied","Data":"43f778e02b84e85d7cb91975890a06bfd388bc908cb159c444d6f512f2515b23"} Sep 16 07:34:00.334041 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.333813 2580 scope.go:117] "RemoveContainer" containerID="d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c" Sep 16 07:34:00.341766 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.341668 2580 scope.go:117] "RemoveContainer" containerID="e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62" Sep 16 07:34:00.348483 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.348468 2580 scope.go:117] "RemoveContainer" containerID="da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4" Sep 16 07:34:00.354860 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.354843 2580 scope.go:117] "RemoveContainer" containerID="e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909" Sep 16 07:34:00.357360 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.357337 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 16 07:34:00.361201 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.361183 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 16 07:34:00.361609 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.361585 2580 scope.go:117] "RemoveContainer" containerID="68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506" Sep 16 07:34:00.367892 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.367876 2580 scope.go:117] "RemoveContainer" containerID="54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4" Sep 16 07:34:00.374324 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.374306 2580 scope.go:117] "RemoveContainer" containerID="444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06" Sep 16 07:34:00.380938 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.380923 2580 scope.go:117] "RemoveContainer" containerID="d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c" Sep 16 07:34:00.381198 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:34:00.381174 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c\": container with ID starting with d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c not found: ID does not exist" containerID="d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c" Sep 16 07:34:00.381271 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.381208 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c"} err="failed to get container status \"d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c\": rpc error: code = NotFound desc = could not find container \"d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c\": container with ID starting with d84686931b888fd6af062a35e0b67df8068f9ff6b8e7b868faaffa9b6181be5c not found: ID does not exist" Sep 16 07:34:00.381271 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.381230 2580 scope.go:117] "RemoveContainer" containerID="e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62" Sep 16 07:34:00.381472 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:34:00.381459 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62\": container with ID starting with e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62 not found: ID does not exist" containerID="e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62" Sep 16 07:34:00.381516 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.381475 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62"} err="failed to get container status \"e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62\": rpc error: code = NotFound desc = could not find container \"e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62\": container with ID starting with e5fa2e7a19630fa10ffc5cdc99683cefa1b8db94f02776652b84e69bbfe62c62 not found: ID does not exist" Sep 16 07:34:00.381516 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.381489 2580 scope.go:117] "RemoveContainer" containerID="da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4" Sep 16 07:34:00.381736 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:34:00.381721 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4\": container with ID starting with da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4 not found: ID does not exist" containerID="da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4" Sep 16 07:34:00.381768 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.381738 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4"} err="failed to get container status \"da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4\": rpc error: code = NotFound desc = could not find container \"da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4\": container with ID starting with da10b08ec5e2af85e1de400ee9388c0ce4468d76d96d929b9e6387fd3f400cd4 not found: ID does not exist" Sep 16 07:34:00.381768 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.381750 2580 scope.go:117] "RemoveContainer" containerID="e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909" Sep 16 07:34:00.381961 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:34:00.381943 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909\": container with ID starting with e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909 not found: ID does not exist" containerID="e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909" Sep 16 07:34:00.381997 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.381964 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909"} err="failed to get container status \"e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909\": rpc error: code = NotFound desc = could not find container \"e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909\": container with ID starting with e12209057273f3ba521947f6d8abeff8d6813767b0c6c5d612e0594cb1c90909 not found: ID does not exist" Sep 16 07:34:00.381997 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.381975 2580 scope.go:117] "RemoveContainer" containerID="68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506" Sep 16 07:34:00.382188 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:34:00.382168 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506\": container with ID starting with 68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506 not found: ID does not exist" containerID="68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506" Sep 16 07:34:00.382232 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.382194 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506"} err="failed to get container status \"68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506\": rpc error: code = NotFound desc = could not find container \"68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506\": container with ID starting with 68e1bc0318a42c1316b0b0e1b02332d9234c296e4c9a037310244472263a4506 not found: ID does not exist" Sep 16 07:34:00.382232 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.382210 2580 scope.go:117] "RemoveContainer" containerID="54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4" Sep 16 07:34:00.382423 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:34:00.382406 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4\": container with ID starting with 54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4 not found: ID does not exist" containerID="54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4" Sep 16 07:34:00.382462 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.382429 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4"} err="failed to get container status \"54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4\": rpc error: code = NotFound desc = could not find container \"54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4\": container with ID starting with 54148ebf96de56db80c70efa09a19ac6f71d06d0bfd57569450d7f704ba5d6d4 not found: ID does not exist" Sep 16 07:34:00.382462 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.382443 2580 scope.go:117] "RemoveContainer" containerID="444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06" Sep 16 07:34:00.382668 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:34:00.382654 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06\": container with ID starting with 444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06 not found: ID does not exist" containerID="444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06" Sep 16 07:34:00.382705 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.382669 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06"} err="failed to get container status \"444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06\": rpc error: code = NotFound desc = could not find container \"444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06\": container with ID starting with 444f917dcc48922fd0fbdabb9189267f7d3f064486ec661fe73147eb085dee06 not found: ID does not exist" Sep 16 07:34:00.388555 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388534 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 16 07:34:00.388821 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388809 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="prom-label-proxy" Sep 16 07:34:00.388861 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388823 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="prom-label-proxy" Sep 16 07:34:00.388861 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388833 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="alertmanager" Sep 16 07:34:00.388861 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388839 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="alertmanager" Sep 16 07:34:00.388861 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388849 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec53573a-703d-4e8e-ab3c-05f3db9253c0" containerName="registry" Sep 16 07:34:00.388861 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388854 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec53573a-703d-4e8e-ab3c-05f3db9253c0" containerName="registry" Sep 16 07:34:00.388861 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388860 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy-web" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388866 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy-web" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388873 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388879 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388886 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="init-config-reloader" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388891 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="init-config-reloader" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388897 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="config-reloader" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388902 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="config-reloader" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388907 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy-metric" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388911 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy-metric" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388956 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy-metric" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388962 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="prom-label-proxy" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388969 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec53573a-703d-4e8e-ab3c-05f3db9253c0" containerName="registry" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388974 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="alertmanager" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388979 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy-web" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388984 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="kube-rbac-proxy" Sep 16 07:34:00.389035 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.388990 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" containerName="config-reloader" Sep 16 07:34:00.392157 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.392141 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.394493 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.394473 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 16 07:34:00.395629 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.395601 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 16 07:34:00.395712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.395685 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 16 07:34:00.395810 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.395755 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 16 07:34:00.395951 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.395812 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 16 07:34:00.395951 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.395928 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r76rv\"" Sep 16 07:34:00.395951 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.395939 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 16 07:34:00.396260 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.396241 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 16 07:34:00.396390 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.396314 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 16 07:34:00.401457 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.401440 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 16 07:34:00.410400 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.410381 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 16 07:34:00.550059 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550059 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvdfr\" (UniqueName: \"kubernetes.io/projected/611c9bef-8477-4b8b-9d87-67f16e43ced4-kube-api-access-kvdfr\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-config-volume\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550145 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-web-config\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/611c9bef-8477-4b8b-9d87-67f16e43ced4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550203 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550223 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/611c9bef-8477-4b8b-9d87-67f16e43ced4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/611c9bef-8477-4b8b-9d87-67f16e43ced4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550504 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/611c9bef-8477-4b8b-9d87-67f16e43ced4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550504 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/611c9bef-8477-4b8b-9d87-67f16e43ced4-config-out\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.550504 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.550314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651504 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651613 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvdfr\" (UniqueName: \"kubernetes.io/projected/611c9bef-8477-4b8b-9d87-67f16e43ced4-kube-api-access-kvdfr\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651613 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-config-volume\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651613 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651613 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-web-config\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/611c9bef-8477-4b8b-9d87-67f16e43ced4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651750 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651983 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/611c9bef-8477-4b8b-9d87-67f16e43ced4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651983 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/611c9bef-8477-4b8b-9d87-67f16e43ced4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651983 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/611c9bef-8477-4b8b-9d87-67f16e43ced4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651983 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/611c9bef-8477-4b8b-9d87-67f16e43ced4-config-out\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.651983 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.651968 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.652963 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.652596 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/611c9bef-8477-4b8b-9d87-67f16e43ced4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.652963 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.652680 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/611c9bef-8477-4b8b-9d87-67f16e43ced4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.653147 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.652986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/611c9bef-8477-4b8b-9d87-67f16e43ced4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.654762 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.654736 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.654899 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.654878 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-web-config\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.655080 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.655012 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.655080 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.655038 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-config-volume\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.655244 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.655224 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.655313 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.655244 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.655467 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.655438 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/611c9bef-8477-4b8b-9d87-67f16e43ced4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.655528 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.655493 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/611c9bef-8477-4b8b-9d87-67f16e43ced4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.656669 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.656648 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/611c9bef-8477-4b8b-9d87-67f16e43ced4-config-out\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.660492 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.660461 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvdfr\" (UniqueName: \"kubernetes.io/projected/611c9bef-8477-4b8b-9d87-67f16e43ced4-kube-api-access-kvdfr\") pod \"alertmanager-main-0\" (UID: \"611c9bef-8477-4b8b-9d87-67f16e43ced4\") " pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.701431 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.701407 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 16 07:34:00.980922 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:00.980865 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 16 07:34:00.983051 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:34:00.983025 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611c9bef_8477_4b8b_9d87_67f16e43ced4.slice/crio-0eeaa6834c9f7d67fcc8b99dda954b93c439a9974b51b708f14bbdb954faf553 WatchSource:0}: Error finding container 0eeaa6834c9f7d67fcc8b99dda954b93c439a9974b51b708f14bbdb954faf553: Status 404 returned error can't find the container with id 0eeaa6834c9f7d67fcc8b99dda954b93c439a9974b51b708f14bbdb954faf553 Sep 16 07:34:01.059175 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:01.059150 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19103d9b-c62c-40c1-adfe-5ffcdad81a10" path="/var/lib/kubelet/pods/19103d9b-c62c-40c1-adfe-5ffcdad81a10/volumes" Sep 16 07:34:01.340141 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:01.340064 2580 generic.go:356] "Generic (PLEG): container finished" podID="611c9bef-8477-4b8b-9d87-67f16e43ced4" containerID="0f1ddab2de324dfdc4162e90ea0c481119f54bd96e2bbdc68ab48e3a7253e4ae" exitCode=0 Sep 16 07:34:01.340141 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:01.340120 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"611c9bef-8477-4b8b-9d87-67f16e43ced4","Type":"ContainerDied","Data":"0f1ddab2de324dfdc4162e90ea0c481119f54bd96e2bbdc68ab48e3a7253e4ae"} Sep 16 07:34:01.340486 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:01.340195 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"611c9bef-8477-4b8b-9d87-67f16e43ced4","Type":"ContainerStarted","Data":"0eeaa6834c9f7d67fcc8b99dda954b93c439a9974b51b708f14bbdb954faf553"} Sep 16 07:34:02.323704 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.323663 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-555c88dcff-x8j6z"] Sep 16 07:34:02.325918 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.325895 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.328288 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.328229 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 16 07:34:02.328288 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.328225 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 16 07:34:02.328288 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.328271 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 16 07:34:02.328288 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.328282 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kth9h\"" Sep 16 07:34:02.328543 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.328410 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 16 07:34:02.328695 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.328681 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 16 07:34:02.333609 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.333587 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 16 07:34:02.343423 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.343396 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-555c88dcff-x8j6z"] Sep 16 07:34:02.349004 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.348982 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"611c9bef-8477-4b8b-9d87-67f16e43ced4","Type":"ContainerStarted","Data":"c1e67d79403beefad094af014707ca3d6c8d4e17a583d6f81f411b0d443ca5c9"} Sep 16 07:34:02.349085 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.349054 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"611c9bef-8477-4b8b-9d87-67f16e43ced4","Type":"ContainerStarted","Data":"5e211d2fc23f4240d5c17bb58a8d5d4f5470b89baa8fb52f364f8521e4fe216c"} Sep 16 07:34:02.349134 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.349088 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"611c9bef-8477-4b8b-9d87-67f16e43ced4","Type":"ContainerStarted","Data":"6bc3afc61697e283b771fc54f0fcdfea09d93b90568bf4c563fea3f675fa0a49"} Sep 16 07:34:02.349134 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.349112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"611c9bef-8477-4b8b-9d87-67f16e43ced4","Type":"ContainerStarted","Data":"c5c536e4f8bec16a9bf60a2c01e5bc220605ab14cc0048988705ab533fdccd29"} Sep 16 07:34:02.349203 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.349139 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"611c9bef-8477-4b8b-9d87-67f16e43ced4","Type":"ContainerStarted","Data":"2af45c78fc62c22cd7fda83c3c792e52a6ad44dafbac2674f7790a12b1a05287"} Sep 16 07:34:02.349203 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.349162 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"611c9bef-8477-4b8b-9d87-67f16e43ced4","Type":"ContainerStarted","Data":"b84c15228afc9fea05e80cdab08a4476b616db38538f654f537beebc4eb873f9"} Sep 16 07:34:02.390737 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.390675 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.3751503 podStartE2EDuration="2.390654363s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.504063ms" isStatefulPod=false podCreationTimestamp="2026-09-16 07:34:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:34:02.38887952 +0000 UTC m=+373.913233298" watchObservedRunningTime="2026-09-16 07:34:02.390654363 +0000 UTC m=+373.915008144" Sep 16 07:34:02.468887 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.468841 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-federate-client-tls\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.469076 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.468932 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dd45bb-834e-415d-a570-1d065b4d1f09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.469076 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.469023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.469237 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.469218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84dd45bb-834e-415d-a570-1d065b4d1f09-metrics-client-ca\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.469286 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.469254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-secret-telemeter-client\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.469371 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.469357 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dd45bb-834e-415d-a570-1d065b4d1f09-serving-certs-ca-bundle\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.469417 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.469382 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9sj6\" (UniqueName: \"kubernetes.io/projected/84dd45bb-834e-415d-a570-1d065b4d1f09-kube-api-access-k9sj6\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.469417 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.469399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-telemeter-client-tls\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.570179 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.570146 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.570318 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.570210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84dd45bb-834e-415d-a570-1d065b4d1f09-metrics-client-ca\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.570318 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.570228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-secret-telemeter-client\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.570318 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.570246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dd45bb-834e-415d-a570-1d065b4d1f09-serving-certs-ca-bundle\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.570318 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.570263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9sj6\" (UniqueName: \"kubernetes.io/projected/84dd45bb-834e-415d-a570-1d065b4d1f09-kube-api-access-k9sj6\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.570318 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.570277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-telemeter-client-tls\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.570318 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.570315 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-federate-client-tls\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.570592 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.570344 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dd45bb-834e-415d-a570-1d065b4d1f09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.571157 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.571125 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84dd45bb-834e-415d-a570-1d065b4d1f09-metrics-client-ca\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.571282 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.571166 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dd45bb-834e-415d-a570-1d065b4d1f09-serving-certs-ca-bundle\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.571538 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.571512 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dd45bb-834e-415d-a570-1d065b4d1f09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.572883 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.572854 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-telemeter-client-tls\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.573001 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.572976 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.573059 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.573020 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-secret-telemeter-client\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.573101 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.573072 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/84dd45bb-834e-415d-a570-1d065b4d1f09-federate-client-tls\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.581392 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.581340 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9sj6\" (UniqueName: \"kubernetes.io/projected/84dd45bb-834e-415d-a570-1d065b4d1f09-kube-api-access-k9sj6\") pod \"telemeter-client-555c88dcff-x8j6z\" (UID: \"84dd45bb-834e-415d-a570-1d065b4d1f09\") " pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.645224 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.645188 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" Sep 16 07:34:02.872355 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.872329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:34:02.874528 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.874509 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1477d2a-2a8c-4b94-b45f-d8d82ece6aab-metrics-certs\") pod \"network-metrics-daemon-99h7c\" (UID: \"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab\") " pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:34:02.919161 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.919135 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-555c88dcff-x8j6z"] Sep 16 07:34:02.922144 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:34:02.922123 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84dd45bb_834e_415d_a570_1d065b4d1f09.slice/crio-c5eac169ba77933e807608c8ba31af63ec8898b5f3de58ece65c9026083a4523 WatchSource:0}: Error finding container c5eac169ba77933e807608c8ba31af63ec8898b5f3de58ece65c9026083a4523: Status 404 returned error can't find the container with id c5eac169ba77933e807608c8ba31af63ec8898b5f3de58ece65c9026083a4523 Sep 16 07:34:02.924074 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:02.924059 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 16 07:34:03.158090 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:03.158024 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xjqjb\"" Sep 16 07:34:03.166722 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:03.166700 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-99h7c" Sep 16 07:34:03.354306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:03.354273 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" event={"ID":"84dd45bb-834e-415d-a570-1d065b4d1f09","Type":"ContainerStarted","Data":"c5eac169ba77933e807608c8ba31af63ec8898b5f3de58ece65c9026083a4523"} Sep 16 07:34:03.431596 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:03.431514 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-99h7c"] Sep 16 07:34:03.434376 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:34:03.434348 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1477d2a_2a8c_4b94_b45f_d8d82ece6aab.slice/crio-4965b19cc8a9e0daa0ec449ac8cc961fda96d71652fda6074f0de7045f84ade5 WatchSource:0}: Error finding container 4965b19cc8a9e0daa0ec449ac8cc961fda96d71652fda6074f0de7045f84ade5: Status 404 returned error can't find the container with id 4965b19cc8a9e0daa0ec449ac8cc961fda96d71652fda6074f0de7045f84ade5 Sep 16 07:34:03.836869 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:03.836762 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:34:03.938100 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:03.938064 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:34:04.365498 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:04.365463 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-99h7c" event={"ID":"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab","Type":"ContainerStarted","Data":"4965b19cc8a9e0daa0ec449ac8cc961fda96d71652fda6074f0de7045f84ade5"} Sep 16 07:34:05.371348 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.371311 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-99h7c" event={"ID":"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab","Type":"ContainerStarted","Data":"5a017983c96a650d7c9cb488beaf196947d008dc4825e0e6ed411d5fbaa781a9"} Sep 16 07:34:05.371733 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.371375 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-99h7c" event={"ID":"e1477d2a-2a8c-4b94-b45f-d8d82ece6aab","Type":"ContainerStarted","Data":"d7bae1250d37cbab5c5a102d7282f80855ff5785616b9429850267e32c9610cf"} Sep 16 07:34:05.373280 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.373257 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" event={"ID":"84dd45bb-834e-415d-a570-1d065b4d1f09","Type":"ContainerStarted","Data":"4e31ac936a85a6c10261fe2121110f82026b2340a394bceedfd47d132aa60e7d"} Sep 16 07:34:05.373387 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.373310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" event={"ID":"84dd45bb-834e-415d-a570-1d065b4d1f09","Type":"ContainerStarted","Data":"3532c14851b444de47b226f1a3f674aa2d3a34e320a12725472d17f3bc2b582f"} Sep 16 07:34:05.373387 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.373337 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" event={"ID":"84dd45bb-834e-415d-a570-1d065b4d1f09","Type":"ContainerStarted","Data":"6ccedbeb5422be94e996fc4befbf4aa201cf33620c569f8cdcbbc99a38a5c860"} Sep 16 07:34:05.387047 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.387011 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-99h7c" podStartSLOduration=374.948074649 podStartE2EDuration="6m16.386998486s" totalImagesPullingTime="1.438923837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:34:05.385740084 +0000 UTC m=+376.910093863" watchObservedRunningTime="2026-09-16 07:34:05.386998486 +0000 UTC m=+376.911352267" Sep 16 07:34:05.407520 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.407481 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-555c88dcff-x8j6z" podStartSLOduration=1.454736899 podStartE2EDuration="3.407468573s" totalImagesPullingTime="1.952731674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:34:05.405913944 +0000 UTC m=+376.930267740" watchObservedRunningTime="2026-09-16 07:34:05.407468573 +0000 UTC m=+376.931822352" Sep 16 07:34:05.487694 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.487664 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:34:05.550510 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.550480 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:34:05.615400 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:05.615371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:34:16.927394 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:16.927357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:34:17.029319 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:17.029285 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:34:17.087558 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:17.087531 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:34:50.719942 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:34:50.719910 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:00.087073 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:00.087026 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:00.191848 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:00.191811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:12.465326 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:12.465244 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:12.562907 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:12.562868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:16.812882 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:16.812838 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:16.872963 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:16.872929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:24.427200 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:24.427128 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:24.479292 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:24.479256 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:28.187488 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:28.187455 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:28.238203 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:28.238154 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:28.324824 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:28.324801 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:33.385524 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:33.385492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:33.485003 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:33.484972 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:33.546288 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:33.546255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:40.988436 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:40.988398 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:35:41.089241 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:35:41.089194 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:37:34.784798 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:34.784755 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:37:35.975012 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:35.974983 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:37:45.639525 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:45.639490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:37:45.690110 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:45.690085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:37:45.745905 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:45.745876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:37:46.357835 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:46.357801 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:37:48.848761 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:48.848735 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:37:48.849733 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:48.849710 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:37:51.169940 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:51.169907 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:37:51.223096 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:37:51.223056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:38:42.778067 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:42.778033 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:38:42.875180 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:42.875152 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:38:43.428548 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.428512 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk"] Sep 16 07:38:43.431758 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.431736 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" Sep 16 07:38:43.433700 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.433679 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 16 07:38:43.433796 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.433751 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 16 07:38:43.434413 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.434394 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-lskk5\"" Sep 16 07:38:43.439261 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.439241 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk"] Sep 16 07:38:43.441736 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.441709 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:38:43.542216 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.542192 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86lvf\" (UniqueName: \"kubernetes.io/projected/3d0cec32-8d3e-4b67-ac58-0b3c7201db22-kube-api-access-86lvf\") pod \"feast-operator-controller-manager-5f77964bcb-4m6lk\" (UID: \"3d0cec32-8d3e-4b67-ac58-0b3c7201db22\") " pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" Sep 16 07:38:43.643125 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.643098 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86lvf\" (UniqueName: \"kubernetes.io/projected/3d0cec32-8d3e-4b67-ac58-0b3c7201db22-kube-api-access-86lvf\") pod \"feast-operator-controller-manager-5f77964bcb-4m6lk\" (UID: \"3d0cec32-8d3e-4b67-ac58-0b3c7201db22\") " pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" Sep 16 07:38:43.650394 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.650368 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86lvf\" (UniqueName: \"kubernetes.io/projected/3d0cec32-8d3e-4b67-ac58-0b3c7201db22-kube-api-access-86lvf\") pod \"feast-operator-controller-manager-5f77964bcb-4m6lk\" (UID: \"3d0cec32-8d3e-4b67-ac58-0b3c7201db22\") " pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" Sep 16 07:38:43.742004 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:43.741953 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" Sep 16 07:38:44.008748 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:44.008728 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk"] Sep 16 07:38:44.011188 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:38:44.011158 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0cec32_8d3e_4b67_ac58_0b3c7201db22.slice/crio-79790ca687dafd7b4a1a1455705107dce80b3981ad58020f7662f82822746538 WatchSource:0}: Error finding container 79790ca687dafd7b4a1a1455705107dce80b3981ad58020f7662f82822746538: Status 404 returned error can't find the container with id 79790ca687dafd7b4a1a1455705107dce80b3981ad58020f7662f82822746538 Sep 16 07:38:44.393775 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:44.393738 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" event={"ID":"3d0cec32-8d3e-4b67-ac58-0b3c7201db22","Type":"ContainerStarted","Data":"79790ca687dafd7b4a1a1455705107dce80b3981ad58020f7662f82822746538"} Sep 16 07:38:47.405888 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:47.405847 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" event={"ID":"3d0cec32-8d3e-4b67-ac58-0b3c7201db22","Type":"ContainerStarted","Data":"1c268ae4c0b413bacf39b1b9e9182c8d54683bef0e5b76ef43c2258efa64b286"} Sep 16 07:38:47.406231 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:47.405955 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" Sep 16 07:38:47.421953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:47.421885 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" podStartSLOduration=2.079003832 podStartE2EDuration="4.4218699s" totalImagesPullingTime="2.342866068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:38:47.42109266 +0000 UTC m=+658.945446442" watchObservedRunningTime="2026-09-16 07:38:47.4218699 +0000 UTC m=+658.946223679" Sep 16 07:38:58.411766 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:38:58.411734 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" Sep 16 07:39:25.428393 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:25.428341 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:39:25.528580 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:25.528553 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:39:26.008224 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.008173 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:39:26.110589 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.110554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:39:26.117761 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.117734 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj"] Sep 16 07:39:26.121267 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.121250 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.123327 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.123293 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 16 07:39:26.123457 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.123330 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-427rh\"" Sep 16 07:39:26.123541 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.123521 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 16 07:39:26.123776 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.123760 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 16 07:39:26.123934 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.123916 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 16 07:39:26.124133 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.123881 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 16 07:39:26.124195 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.123916 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 16 07:39:26.124256 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.123856 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 16 07:39:26.131931 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.131910 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj"] Sep 16 07:39:26.263131 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.263059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-online-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.263255 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.263137 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77688205-8031-40cc-befb-3b563880a4f7-feast-data\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.263255 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.263168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-registry-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.263255 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.263193 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-ui-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.263255 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.263222 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-offline-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.263464 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.263255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmq6n\" (UniqueName: \"kubernetes.io/projected/77688205-8031-40cc-befb-3b563880a4f7-kube-api-access-dmq6n\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.363733 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.363707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-online-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.363891 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.363770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77688205-8031-40cc-befb-3b563880a4f7-feast-data\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.363891 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.363827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-registry-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.363891 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.363868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-ui-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.364075 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:39:26.363944 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 16 07:39:26.364075 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:39:26.364014 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-registry-tls podName:77688205-8031-40cc-befb-3b563880a4f7 nodeName:}" failed. No retries permitted until 2026-09-16 07:39:26.863993976 +0000 UTC m=+698.388347752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-registry-tls") pod "feast-simple-feast-setup-6b8d6dbb67-mjvxj" (UID: "77688205-8031-40cc-befb-3b563880a4f7") : secret "feast-simple-feast-setup-registry-tls" not found Sep 16 07:39:26.364075 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.364030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-offline-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.364075 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.364063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmq6n\" (UniqueName: \"kubernetes.io/projected/77688205-8031-40cc-befb-3b563880a4f7-kube-api-access-dmq6n\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.364290 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.364218 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77688205-8031-40cc-befb-3b563880a4f7-feast-data\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.364290 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:39:26.364282 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 16 07:39:26.364389 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:39:26.364350 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-offline-tls podName:77688205-8031-40cc-befb-3b563880a4f7 nodeName:}" failed. No retries permitted until 2026-09-16 07:39:26.864331132 +0000 UTC m=+698.388684898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-offline-tls") pod "feast-simple-feast-setup-6b8d6dbb67-mjvxj" (UID: "77688205-8031-40cc-befb-3b563880a4f7") : secret "feast-simple-feast-setup-offline-tls" not found Sep 16 07:39:26.366278 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.366251 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-online-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.366385 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.366354 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-ui-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.368205 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.368175 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:39:26.371953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.371931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmq6n\" (UniqueName: \"kubernetes.io/projected/77688205-8031-40cc-befb-3b563880a4f7-kube-api-access-dmq6n\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.468388 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.468355 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:39:26.566642 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.566586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:39:26.667654 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.667620 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:39:26.867406 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.867379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-registry-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.867537 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.867412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-offline-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.869959 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.869930 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-registry-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:26.870075 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:26.869930 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-offline-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-mjvxj\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:27.044860 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:27.044829 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:39:27.320492 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:27.320424 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj"] Sep 16 07:39:27.323301 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:39:27.323275 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77688205_8031_40cc_befb_3b563880a4f7.slice/crio-1858d90f35a2a11e5afd713af1aec8552a1de8e1a613636c093984131794950f WatchSource:0}: Error finding container 1858d90f35a2a11e5afd713af1aec8552a1de8e1a613636c093984131794950f: Status 404 returned error can't find the container with id 1858d90f35a2a11e5afd713af1aec8552a1de8e1a613636c093984131794950f Sep 16 07:39:27.325219 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:27.325199 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 16 07:39:27.558071 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:27.558037 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerStarted","Data":"1858d90f35a2a11e5afd713af1aec8552a1de8e1a613636c093984131794950f"} Sep 16 07:39:52.679798 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:52.679700 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerStarted","Data":"699ca1a2157dca0c5b0e3b5611019d3f5105c3a02b5ef69a4f89d762b702df0b"} Sep 16 07:39:57.700277 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:57.700244 2580 generic.go:356] "Generic (PLEG): container finished" podID="77688205-8031-40cc-befb-3b563880a4f7" containerID="699ca1a2157dca0c5b0e3b5611019d3f5105c3a02b5ef69a4f89d762b702df0b" exitCode=0 Sep 16 07:39:57.700728 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:57.700290 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerDied","Data":"699ca1a2157dca0c5b0e3b5611019d3f5105c3a02b5ef69a4f89d762b702df0b"} Sep 16 07:39:58.705961 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:39:58.705925 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerStarted","Data":"4ef083c8b988960a6dc4c490c5bf96fd83cf0e104ee98ea8ef8f907e4a34bcfe"} Sep 16 07:40:03.728980 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:03.728941 2580 generic.go:356] "Generic (PLEG): container finished" podID="77688205-8031-40cc-befb-3b563880a4f7" containerID="4ef083c8b988960a6dc4c490c5bf96fd83cf0e104ee98ea8ef8f907e4a34bcfe" exitCode=0 Sep 16 07:40:03.729282 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:03.729015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerDied","Data":"4ef083c8b988960a6dc4c490c5bf96fd83cf0e104ee98ea8ef8f907e4a34bcfe"} Sep 16 07:40:03.738519 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:03.738501 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 16 07:40:04.739595 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:04.739551 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerStarted","Data":"26b19a5b5b09948eb2bf53a85da73f9eb7cf9d6a2e111f78c761b2ce669c384d"} Sep 16 07:40:04.740121 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:04.739606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerStarted","Data":"a6c74ad3baa43987e5d166a06304cb16fd9ae55da26b7a250849e1d73b70d92f"} Sep 16 07:40:04.740121 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:04.739629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerStarted","Data":"c9d8684755e8c93bd478bf109ae0a64bacd0578f43c1d04faf69185ad47c530a"} Sep 16 07:40:04.740121 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:04.739649 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerStarted","Data":"9eddd7793ff0e4d8bcb54be53b4be124fc20035df0cc4facfad81d62e7cf6b38"} Sep 16 07:40:04.766642 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:04.766571 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podStartSLOduration=13.689715649 podStartE2EDuration="38.766548764s" totalImagesPullingTime="25.076833115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:40:04.763934639 +0000 UTC m=+736.288288418" watchObservedRunningTime="2026-09-16 07:40:04.766548764 +0000 UTC m=+736.290902544" Sep 16 07:40:06.045953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:06.045843 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:06.045953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:06.045914 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:06.045953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:06.045934 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:06.045953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:06.045952 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:06.048234 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:06.048190 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 16 07:40:06.048234 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:06.048205 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Sep 16 07:40:06.048437 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:06.048237 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused" Sep 16 07:40:06.048437 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:06.048292 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 16 07:40:07.045326 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:07.045289 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:07.045577 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:07.045554 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:07.045577 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:07.045581 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:07.045804 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:07.045603 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:09.045565 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:09.045479 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 16 07:40:09.046077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:09.045508 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused" Sep 16 07:40:09.046077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:09.045507 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Sep 16 07:40:09.046330 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:09.046299 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 16 07:40:12.046417 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:12.046383 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:12.046835 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:12.046560 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:12.047091 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:12.047067 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:12.047262 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:12.047237 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:12.047424 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:12.047407 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:12.051846 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:12.051828 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:12.412270 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:12.412242 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:40:12.783408 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:12.783322 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:12.786315 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:12.786286 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:40:13.270561 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:13.270529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:40:50.016149 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:40:50.016115 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:14.616082 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:14.615992 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:14.622242 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:14.622212 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj"] Sep 16 07:41:14.622491 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:14.622442 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="registry" containerID="cri-o://9eddd7793ff0e4d8bcb54be53b4be124fc20035df0cc4facfad81d62e7cf6b38" gracePeriod=30 Sep 16 07:41:14.622616 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:14.622573 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="offline" containerID="cri-o://a6c74ad3baa43987e5d166a06304cb16fd9ae55da26b7a250849e1d73b70d92f" gracePeriod=30 Sep 16 07:41:14.622716 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:14.622688 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="online" containerID="cri-o://c9d8684755e8c93bd478bf109ae0a64bacd0578f43c1d04faf69185ad47c530a" gracePeriod=30 Sep 16 07:41:14.622858 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:14.622716 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="ui" containerID="cri-o://26b19a5b5b09948eb2bf53a85da73f9eb7cf9d6a2e111f78c761b2ce669c384d" gracePeriod=30 Sep 16 07:41:14.711242 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:14.711211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:14.883396 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:14.883295 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:14.977933 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:14.977821 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:15.029806 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.029762 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr"] Sep 16 07:41:15.035049 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.034985 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.043593 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.039434 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr"] Sep 16 07:41:15.043593 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.039723 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-zggtj\"" Sep 16 07:41:15.100360 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.100321 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-online-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.100658 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.100642 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-registry-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.100779 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.100766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-feast-data\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.100967 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.100953 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-ui-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.101091 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.101076 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pt9c\" (UniqueName: \"kubernetes.io/projected/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-kube-api-access-2pt9c\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.101209 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.101197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-offline-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.202682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-ui-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.202738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pt9c\" (UniqueName: \"kubernetes.io/projected/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-kube-api-access-2pt9c\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.202774 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-offline-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.202839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-online-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.202894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-registry-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.202915 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-feast-data\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:41:15.203154 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:41:15.203231 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-online-tls podName:a0eb47ab-61c0-41fe-9f4e-70ebed867c36 nodeName:}" failed. No retries permitted until 2026-09-16 07:41:15.70321002 +0000 UTC m=+807.227563795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-online-tls") pod "feast-simple-feast-setup-6b8d6dbb67-9h2pr" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36") : secret "feast-simple-feast-setup-online-tls" not found Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.203369 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-feast-data\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:41:15.203427 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:41:15.203463 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-offline-tls podName:a0eb47ab-61c0-41fe-9f4e-70ebed867c36 nodeName:}" failed. No retries permitted until 2026-09-16 07:41:15.703451509 +0000 UTC m=+807.227805281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-offline-tls") pod "feast-simple-feast-setup-6b8d6dbb67-9h2pr" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36") : secret "feast-simple-feast-setup-offline-tls" not found Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:41:15.203470 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:41:15.203528 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-registry-tls podName:a0eb47ab-61c0-41fe-9f4e-70ebed867c36 nodeName:}" failed. No retries permitted until 2026-09-16 07:41:15.703511624 +0000 UTC m=+807.227865397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-registry-tls") pod "feast-simple-feast-setup-6b8d6dbb67-9h2pr" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36") : secret "feast-simple-feast-setup-registry-tls" not found Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:41:15.203532 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 16 07:41:15.203603 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:41:15.203589 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-ui-tls podName:a0eb47ab-61c0-41fe-9f4e-70ebed867c36 nodeName:}" failed. No retries permitted until 2026-09-16 07:41:15.703578559 +0000 UTC m=+807.227932321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-ui-tls") pod "feast-simple-feast-setup-6b8d6dbb67-9h2pr" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36") : secret "feast-simple-feast-setup-ui-tls" not found Sep 16 07:41:15.213444 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.213389 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pt9c\" (UniqueName: \"kubernetes.io/projected/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-kube-api-access-2pt9c\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.333755 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.333722 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:15.467599 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.467544 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:15.567810 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.567769 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:15.692666 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.692638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:15.707661 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.707633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-online-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.707777 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.707691 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-registry-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.707777 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.707758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-ui-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.707918 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.707822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-offline-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.710262 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.710234 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-offline-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.710334 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.710270 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-registry-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.710334 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.710278 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-online-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.710409 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.710378 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-ui-tls\") pod \"feast-simple-feast-setup-6b8d6dbb67-9h2pr\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:15.952670 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:15.952634 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:16.036433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:16.036408 2580 generic.go:356] "Generic (PLEG): container finished" podID="77688205-8031-40cc-befb-3b563880a4f7" containerID="26b19a5b5b09948eb2bf53a85da73f9eb7cf9d6a2e111f78c761b2ce669c384d" exitCode=0 Sep 16 07:41:16.036433 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:16.036429 2580 generic.go:356] "Generic (PLEG): container finished" podID="77688205-8031-40cc-befb-3b563880a4f7" containerID="c9d8684755e8c93bd478bf109ae0a64bacd0578f43c1d04faf69185ad47c530a" exitCode=0 Sep 16 07:41:16.036580 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:16.036485 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerDied","Data":"26b19a5b5b09948eb2bf53a85da73f9eb7cf9d6a2e111f78c761b2ce669c384d"} Sep 16 07:41:16.036580 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:16.036534 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerDied","Data":"c9d8684755e8c93bd478bf109ae0a64bacd0578f43c1d04faf69185ad47c530a"} Sep 16 07:41:16.228209 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:16.228187 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr"] Sep 16 07:41:16.229955 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:41:16.229931 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0eb47ab_61c0_41fe_9f4e_70ebed867c36.slice/crio-3b544cbcff6a6cc0da2f85a6fe629ef35ef890c3d5e2fa45503505f4d8459097 WatchSource:0}: Error finding container 3b544cbcff6a6cc0da2f85a6fe629ef35ef890c3d5e2fa45503505f4d8459097: Status 404 returned error can't find the container with id 3b544cbcff6a6cc0da2f85a6fe629ef35ef890c3d5e2fa45503505f4d8459097 Sep 16 07:41:17.041703 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:17.041664 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerStarted","Data":"87a4316f40369339e743e45900b2e8060d8341eb8351bd3acbd94b33a6e9c19d"} Sep 16 07:41:17.042090 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:17.041710 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerStarted","Data":"3b544cbcff6a6cc0da2f85a6fe629ef35ef890c3d5e2fa45503505f4d8459097"} Sep 16 07:41:21.060124 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:21.060100 2580 generic.go:356] "Generic (PLEG): container finished" podID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerID="87a4316f40369339e743e45900b2e8060d8341eb8351bd3acbd94b33a6e9c19d" exitCode=0 Sep 16 07:41:21.060423 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:21.060400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerDied","Data":"87a4316f40369339e743e45900b2e8060d8341eb8351bd3acbd94b33a6e9c19d"} Sep 16 07:41:22.047604 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:22.047559 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 16 07:41:22.065801 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:22.065755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerStarted","Data":"9fc9080f0c2141f48a100dc667aa81e39cd1a3afb8d3af2a4157dddeffe6130b"} Sep 16 07:41:22.783835 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:22.783772 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 16 07:41:27.086825 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:27.086740 2580 generic.go:356] "Generic (PLEG): container finished" podID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerID="9fc9080f0c2141f48a100dc667aa81e39cd1a3afb8d3af2a4157dddeffe6130b" exitCode=0 Sep 16 07:41:27.086825 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:27.086812 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerDied","Data":"9fc9080f0c2141f48a100dc667aa81e39cd1a3afb8d3af2a4157dddeffe6130b"} Sep 16 07:41:28.097857 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:28.097817 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerStarted","Data":"5d86371c94294cf41cb677e8dc5819cf07955c142c3e3a0da6acd6a09a8adca6"} Sep 16 07:41:28.098324 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:28.097883 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerStarted","Data":"e776e1d4d551b2b51c1586bf62e843025eae82f8631ce3d59608d85c75416e4b"} Sep 16 07:41:28.098324 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:28.097907 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerStarted","Data":"9a751b7db15cf9ba3bfa3e20c5ef603a6728a169b30b9f755267371d018cf2b5"} Sep 16 07:41:28.098324 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:28.097926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerStarted","Data":"02aebbd376bdd4fd0eedfb0d1f919c92963d89d6ad9185aacfa33a310490c77a"} Sep 16 07:41:28.122758 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:28.122696 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podStartSLOduration=13.122672732 podStartE2EDuration="13.122672732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:41:28.121254127 +0000 UTC m=+819.645607908" watchObservedRunningTime="2026-09-16 07:41:28.122672732 +0000 UTC m=+819.647026512" Sep 16 07:41:30.953468 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:30.953427 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:30.954078 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:30.954057 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:30.954210 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:30.954198 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:30.954519 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:30.954486 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:30.956144 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:30.955985 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 16 07:41:30.956144 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:30.956008 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 16 07:41:30.956144 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:30.956016 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 16 07:41:30.956144 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:30.956066 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 16 07:41:32.047371 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:32.047319 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 16 07:41:32.783266 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:32.783214 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 16 07:41:33.953888 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:33.953819 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 16 07:41:33.953888 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:33.953842 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 16 07:41:33.953888 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:33.953844 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 16 07:41:33.954487 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:33.953880 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 16 07:41:35.953112 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:35.953072 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:35.953681 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:35.953134 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:35.953681 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:35.953154 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:35.953681 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:35.953170 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:36.953581 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:36.953542 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:36.954134 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:36.954111 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:36.954315 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:36.954300 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:36.954376 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:36.954359 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:36.958941 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:36.958925 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:37.144961 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:37.144935 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:37.145116 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:37.145002 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:37.147632 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:37.147606 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:41:37.466925 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:37.466891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:41:42.047311 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:42.047270 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Sep 16 07:41:42.047698 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:42.047388 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:41:42.783854 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:42.783816 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 16 07:41:42.784813 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:42.784767 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:41:45.180086 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.180057 2580 generic.go:356] "Generic (PLEG): container finished" podID="77688205-8031-40cc-befb-3b563880a4f7" containerID="a6c74ad3baa43987e5d166a06304cb16fd9ae55da26b7a250849e1d73b70d92f" exitCode=137 Sep 16 07:41:45.180086 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.180086 2580 generic.go:356] "Generic (PLEG): container finished" podID="77688205-8031-40cc-befb-3b563880a4f7" containerID="9eddd7793ff0e4d8bcb54be53b4be124fc20035df0cc4facfad81d62e7cf6b38" exitCode=137 Sep 16 07:41:45.180449 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.180136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerDied","Data":"a6c74ad3baa43987e5d166a06304cb16fd9ae55da26b7a250849e1d73b70d92f"} Sep 16 07:41:45.180449 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.180194 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerDied","Data":"9eddd7793ff0e4d8bcb54be53b4be124fc20035df0cc4facfad81d62e7cf6b38"} Sep 16 07:41:45.267174 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.267153 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:41:45.345698 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.345672 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-online-tls\") pod \"77688205-8031-40cc-befb-3b563880a4f7\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " Sep 16 07:41:45.345828 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.345713 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-ui-tls\") pod \"77688205-8031-40cc-befb-3b563880a4f7\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " Sep 16 07:41:45.345828 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.345764 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-registry-tls\") pod \"77688205-8031-40cc-befb-3b563880a4f7\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " Sep 16 07:41:45.345828 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.345778 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-offline-tls\") pod \"77688205-8031-40cc-befb-3b563880a4f7\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " Sep 16 07:41:45.345828 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.345818 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmq6n\" (UniqueName: \"kubernetes.io/projected/77688205-8031-40cc-befb-3b563880a4f7-kube-api-access-dmq6n\") pod \"77688205-8031-40cc-befb-3b563880a4f7\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " Sep 16 07:41:45.346066 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.345854 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77688205-8031-40cc-befb-3b563880a4f7-feast-data\") pod \"77688205-8031-40cc-befb-3b563880a4f7\" (UID: \"77688205-8031-40cc-befb-3b563880a4f7\") " Sep 16 07:41:45.346427 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.346390 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77688205-8031-40cc-befb-3b563880a4f7-feast-data" (OuterVolumeSpecName: "feast-data") pod "77688205-8031-40cc-befb-3b563880a4f7" (UID: "77688205-8031-40cc-befb-3b563880a4f7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:41:45.347952 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.347931 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "77688205-8031-40cc-befb-3b563880a4f7" (UID: "77688205-8031-40cc-befb-3b563880a4f7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:41:45.348279 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.348243 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-online-tls" (OuterVolumeSpecName: "online-tls") pod "77688205-8031-40cc-befb-3b563880a4f7" (UID: "77688205-8031-40cc-befb-3b563880a4f7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:41:45.348380 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.348309 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "77688205-8031-40cc-befb-3b563880a4f7" (UID: "77688205-8031-40cc-befb-3b563880a4f7"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:41:45.348380 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.348319 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77688205-8031-40cc-befb-3b563880a4f7-kube-api-access-dmq6n" (OuterVolumeSpecName: "kube-api-access-dmq6n") pod "77688205-8031-40cc-befb-3b563880a4f7" (UID: "77688205-8031-40cc-befb-3b563880a4f7"). InnerVolumeSpecName "kube-api-access-dmq6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:41:45.348380 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.348336 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "77688205-8031-40cc-befb-3b563880a4f7" (UID: "77688205-8031-40cc-befb-3b563880a4f7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:41:45.446963 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.446895 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77688205-8031-40cc-befb-3b563880a4f7-feast-data\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:41:45.446963 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.446952 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-online-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:41:45.446963 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.446966 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-ui-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:41:45.447119 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.446978 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-registry-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:41:45.447119 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.446989 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77688205-8031-40cc-befb-3b563880a4f7-offline-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:41:45.447119 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:45.447001 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmq6n\" (UniqueName: \"kubernetes.io/projected/77688205-8031-40cc-befb-3b563880a4f7-kube-api-access-dmq6n\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:41:46.188021 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.187996 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" Sep 16 07:41:46.188417 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.187995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj" event={"ID":"77688205-8031-40cc-befb-3b563880a4f7","Type":"ContainerDied","Data":"1858d90f35a2a11e5afd713af1aec8552a1de8e1a613636c093984131794950f"} Sep 16 07:41:46.188417 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.188119 2580 scope.go:117] "RemoveContainer" containerID="26b19a5b5b09948eb2bf53a85da73f9eb7cf9d6a2e111f78c761b2ce669c384d" Sep 16 07:41:46.196618 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.196602 2580 scope.go:117] "RemoveContainer" containerID="a6c74ad3baa43987e5d166a06304cb16fd9ae55da26b7a250849e1d73b70d92f" Sep 16 07:41:46.204311 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.204294 2580 scope.go:117] "RemoveContainer" containerID="c9d8684755e8c93bd478bf109ae0a64bacd0578f43c1d04faf69185ad47c530a" Sep 16 07:41:46.210448 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.210424 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj"] Sep 16 07:41:46.212511 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.212493 2580 scope.go:117] "RemoveContainer" containerID="9eddd7793ff0e4d8bcb54be53b4be124fc20035df0cc4facfad81d62e7cf6b38" Sep 16 07:41:46.214032 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.214014 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-mjvxj"] Sep 16 07:41:46.219768 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.219749 2580 scope.go:117] "RemoveContainer" containerID="4ef083c8b988960a6dc4c490c5bf96fd83cf0e104ee98ea8ef8f907e4a34bcfe" Sep 16 07:41:46.228302 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:46.228277 2580 scope.go:117] "RemoveContainer" containerID="699ca1a2157dca0c5b0e3b5611019d3f5105c3a02b5ef69a4f89d762b702df0b" Sep 16 07:41:47.059240 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:41:47.059209 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77688205-8031-40cc-befb-3b563880a4f7" path="/var/lib/kubelet/pods/77688205-8031-40cc-befb-3b563880a4f7/volumes" Sep 16 07:42:39.571921 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:39.571874 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:42:40.181866 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.181830 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d"] Sep 16 07:42:40.182738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.182715 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="feast-init" Sep 16 07:42:40.182890 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.182878 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="feast-init" Sep 16 07:42:40.182985 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.182975 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="offline" Sep 16 07:42:40.183056 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183048 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="offline" Sep 16 07:42:40.183137 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183128 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="feast-apply" Sep 16 07:42:40.183198 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183191 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="feast-apply" Sep 16 07:42:40.183264 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183256 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="online" Sep 16 07:42:40.183324 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183316 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="online" Sep 16 07:42:40.183407 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183398 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="registry" Sep 16 07:42:40.183473 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183466 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="registry" Sep 16 07:42:40.183546 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183538 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="ui" Sep 16 07:42:40.183606 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183599 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="ui" Sep 16 07:42:40.183835 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183822 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="ui" Sep 16 07:42:40.183933 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183924 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="registry" Sep 16 07:42:40.184000 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.183992 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="offline" Sep 16 07:42:40.184065 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.184057 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77688205-8031-40cc-befb-3b563880a4f7" containerName="online" Sep 16 07:42:40.188952 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.188930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.200516 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.200321 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 16 07:42:40.200516 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.200357 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 16 07:42:40.201087 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.200683 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 16 07:42:40.201087 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.200690 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 16 07:42:40.201087 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.200975 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 16 07:42:40.201258 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.201217 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 16 07:42:40.201532 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.201506 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-5z75m\"" Sep 16 07:42:40.202246 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.201714 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 16 07:42:40.209824 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.209802 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d"] Sep 16 07:42:40.235566 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.235535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-ui-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.235664 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.235600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-online-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.235664 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.235646 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-offline-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.235825 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.235677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/48ce88e1-39e7-4265-bc28-be77b2721f46-registry-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.235825 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.235702 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/48ce88e1-39e7-4265-bc28-be77b2721f46-feast-data\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.235825 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.235737 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b56x4\" (UniqueName: \"kubernetes.io/projected/48ce88e1-39e7-4265-bc28-be77b2721f46-kube-api-access-b56x4\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.336564 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.336536 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/48ce88e1-39e7-4265-bc28-be77b2721f46-registry-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.336664 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.336584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/48ce88e1-39e7-4265-bc28-be77b2721f46-feast-data\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.336664 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.336619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b56x4\" (UniqueName: \"kubernetes.io/projected/48ce88e1-39e7-4265-bc28-be77b2721f46-kube-api-access-b56x4\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.336760 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.336666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-ui-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.336760 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.336702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-online-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.336760 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.336728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-offline-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.336929 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:42:40.336805 2580 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 16 07:42:40.336929 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:42:40.336875 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-ui-tls podName:48ce88e1-39e7-4265-bc28-be77b2721f46 nodeName:}" failed. No retries permitted until 2026-09-16 07:42:40.836853399 +0000 UTC m=+892.361207160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-ui-tls") pod "feast-simple-feast-remote-setup-7679d98b45-hrd5d" (UID: "48ce88e1-39e7-4265-bc28-be77b2721f46") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 16 07:42:40.337074 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.337029 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/48ce88e1-39e7-4265-bc28-be77b2721f46-feast-data\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.337204 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:42:40.337162 2580 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 16 07:42:40.337272 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:42:40.337232 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-online-tls podName:48ce88e1-39e7-4265-bc28-be77b2721f46 nodeName:}" failed. No retries permitted until 2026-09-16 07:42:40.837214445 +0000 UTC m=+892.361568206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-online-tls") pod "feast-simple-feast-remote-setup-7679d98b45-hrd5d" (UID: "48ce88e1-39e7-4265-bc28-be77b2721f46") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 16 07:42:40.337355 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.337333 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/48ce88e1-39e7-4265-bc28-be77b2721f46-registry-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.339137 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.339116 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-offline-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.346229 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.346205 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b56x4\" (UniqueName: \"kubernetes.io/projected/48ce88e1-39e7-4265-bc28-be77b2721f46-kube-api-access-b56x4\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.841242 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.841212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-ui-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.841617 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.841253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-online-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.843671 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.843646 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-ui-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.843795 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.843763 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-online-tls\") pod \"feast-simple-feast-remote-setup-7679d98b45-hrd5d\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:40.962996 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:40.962954 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:42:41.123490 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:41.123461 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:42:41.412064 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:41.412042 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d"] Sep 16 07:42:41.413840 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:42:41.413813 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48ce88e1_39e7_4265_bc28_be77b2721f46.slice/crio-94bd1b6bb215c1eab2a5cfae75cd6bdbdd5361d03cb0b90e018aa7200e21b794 WatchSource:0}: Error finding container 94bd1b6bb215c1eab2a5cfae75cd6bdbdd5361d03cb0b90e018aa7200e21b794: Status 404 returned error can't find the container with id 94bd1b6bb215c1eab2a5cfae75cd6bdbdd5361d03cb0b90e018aa7200e21b794 Sep 16 07:42:42.401838 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:42.401781 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerStarted","Data":"a0683226abe602762a76bd0acfda3542f3c66904c31a8f991faf05b00e233bd6"} Sep 16 07:42:42.402231 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:42.401850 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerStarted","Data":"94bd1b6bb215c1eab2a5cfae75cd6bdbdd5361d03cb0b90e018aa7200e21b794"} Sep 16 07:42:46.419826 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:46.419774 2580 generic.go:356] "Generic (PLEG): container finished" podID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerID="a0683226abe602762a76bd0acfda3542f3c66904c31a8f991faf05b00e233bd6" exitCode=0 Sep 16 07:42:46.420201 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:46.419849 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerDied","Data":"a0683226abe602762a76bd0acfda3542f3c66904c31a8f991faf05b00e233bd6"} Sep 16 07:42:47.426224 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:47.426190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerStarted","Data":"20f93aa60720a783ef0456365acfb240f21c39dfca5ddcd432f2290ba381f4fc"} Sep 16 07:42:48.880224 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:48.880194 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:42:48.881669 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:48.881648 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:42:59.443531 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:42:59.443500 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:43:08.512398 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:08.512359 2580 generic.go:356] "Generic (PLEG): container finished" podID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerID="20f93aa60720a783ef0456365acfb240f21c39dfca5ddcd432f2290ba381f4fc" exitCode=0 Sep 16 07:43:08.512902 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:08.512422 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerDied","Data":"20f93aa60720a783ef0456365acfb240f21c39dfca5ddcd432f2290ba381f4fc"} Sep 16 07:43:08.517137 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:08.517103 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 16 07:43:09.522221 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:09.522180 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerStarted","Data":"66754faed7532d585ac877532dd1190763355f8b7be9f8b1340f97d7ee0f4679"} Sep 16 07:43:09.522583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:09.522245 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerStarted","Data":"91a21c0516f1eaf855305b95bd66fcb5911bbb64a4b8782544591bfa4210ce64"} Sep 16 07:43:09.522583 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:09.522266 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerStarted","Data":"3b04b2caf3dade44978aad97d85b5b054046fa0810663c3d625920fd31071ca5"} Sep 16 07:43:09.544773 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:09.544719 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podStartSLOduration=29.544698294 podStartE2EDuration="29.544698294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:43:09.543204958 +0000 UTC m=+921.067558749" watchObservedRunningTime="2026-09-16 07:43:09.544698294 +0000 UTC m=+921.069052073" Sep 16 07:43:11.123876 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:11.123840 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:11.124371 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:11.124085 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:11.124371 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:11.124116 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:11.124371 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:11.124132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:11.124371 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:11.124147 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:11.124371 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:11.124162 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:11.125924 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:11.125885 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 16 07:43:11.126065 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:11.125920 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 16 07:43:11.126065 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:11.125920 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Sep 16 07:43:14.123953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:14.123904 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Sep 16 07:43:14.123953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:14.123932 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 16 07:43:14.124371 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:14.123914 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 16 07:43:17.124301 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:17.124273 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:17.124920 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:17.124765 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:17.125043 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:17.125025 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:17.145600 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:17.145577 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:17.564013 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:17.563930 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:17.583397 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:17.583371 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:43:17.952715 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:43:17.952686 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:44:20.347419 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.347328 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:44:20.353276 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.353248 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d"] Sep 16 07:44:20.353643 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.353517 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="online" containerID="cri-o://3b04b2caf3dade44978aad97d85b5b054046fa0810663c3d625920fd31071ca5" gracePeriod=30 Sep 16 07:44:20.353643 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.353567 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="offline" containerID="cri-o://91a21c0516f1eaf855305b95bd66fcb5911bbb64a4b8782544591bfa4210ce64" gracePeriod=30 Sep 16 07:44:20.353831 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.353566 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="ui" containerID="cri-o://66754faed7532d585ac877532dd1190763355f8b7be9f8b1340f97d7ee0f4679" gracePeriod=30 Sep 16 07:44:20.408563 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.408535 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:44:20.471911 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.471880 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr"] Sep 16 07:44:20.472188 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.472157 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="registry" containerID="cri-o://02aebbd376bdd4fd0eedfb0d1f919c92963d89d6ad9185aacfa33a310490c77a" gracePeriod=30 Sep 16 07:44:20.472279 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.472194 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="offline" containerID="cri-o://e776e1d4d551b2b51c1586bf62e843025eae82f8631ce3d59608d85c75416e4b" gracePeriod=30 Sep 16 07:44:20.472279 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.472236 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="online" containerID="cri-o://9a751b7db15cf9ba3bfa3e20c5ef603a6728a169b30b9f755267371d018cf2b5" gracePeriod=30 Sep 16 07:44:20.472279 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.472239 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="ui" containerID="cri-o://5d86371c94294cf41cb677e8dc5819cf07955c142c3e3a0da6acd6a09a8adca6" gracePeriod=30 Sep 16 07:44:20.480007 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.479985 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:44:20.565155 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:20.565125 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:44:21.820453 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:21.820422 2580 generic.go:356] "Generic (PLEG): container finished" podID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerID="66754faed7532d585ac877532dd1190763355f8b7be9f8b1340f97d7ee0f4679" exitCode=0 Sep 16 07:44:21.820453 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:21.820447 2580 generic.go:356] "Generic (PLEG): container finished" podID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerID="3b04b2caf3dade44978aad97d85b5b054046fa0810663c3d625920fd31071ca5" exitCode=0 Sep 16 07:44:21.820916 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:21.820495 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerDied","Data":"66754faed7532d585ac877532dd1190763355f8b7be9f8b1340f97d7ee0f4679"} Sep 16 07:44:21.820916 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:21.820535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerDied","Data":"3b04b2caf3dade44978aad97d85b5b054046fa0810663c3d625920fd31071ca5"} Sep 16 07:44:21.823396 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:21.823375 2580 generic.go:356] "Generic (PLEG): container finished" podID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerID="5d86371c94294cf41cb677e8dc5819cf07955c142c3e3a0da6acd6a09a8adca6" exitCode=0 Sep 16 07:44:21.823396 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:21.823393 2580 generic.go:356] "Generic (PLEG): container finished" podID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerID="9a751b7db15cf9ba3bfa3e20c5ef603a6728a169b30b9f755267371d018cf2b5" exitCode=0 Sep 16 07:44:21.823522 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:21.823432 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerDied","Data":"5d86371c94294cf41cb677e8dc5819cf07955c142c3e3a0da6acd6a09a8adca6"} Sep 16 07:44:21.823522 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:21.823457 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerDied","Data":"9a751b7db15cf9ba3bfa3e20c5ef603a6728a169b30b9f755267371d018cf2b5"} Sep 16 07:44:25.636162 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:25.636123 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:44:27.125511 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:27.125472 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 16 07:44:27.144977 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:27.144946 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 16 07:44:27.145099 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:27.145051 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 16 07:44:27.563952 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:27.563857 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 16 07:44:37.124917 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:37.124879 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 16 07:44:37.144538 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:37.144502 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 16 07:44:37.144673 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:37.144538 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 16 07:44:37.564313 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:37.564224 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 16 07:44:47.125485 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:47.125445 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 16 07:44:47.125881 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:47.125559 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:44:47.144875 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:47.144843 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 16 07:44:47.144996 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:47.144888 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 16 07:44:47.144996 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:47.144973 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:44:47.145105 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:47.145025 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:44:47.564726 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:47.564614 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 16 07:44:47.564921 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:47.564758 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:44:50.943436 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:50.943401 2580 generic.go:356] "Generic (PLEG): container finished" podID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerID="91a21c0516f1eaf855305b95bd66fcb5911bbb64a4b8782544591bfa4210ce64" exitCode=137 Sep 16 07:44:50.943776 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:50.943454 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerDied","Data":"91a21c0516f1eaf855305b95bd66fcb5911bbb64a4b8782544591bfa4210ce64"} Sep 16 07:44:50.947045 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:50.947020 2580 generic.go:356] "Generic (PLEG): container finished" podID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerID="e776e1d4d551b2b51c1586bf62e843025eae82f8631ce3d59608d85c75416e4b" exitCode=137 Sep 16 07:44:50.947045 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:50.947046 2580 generic.go:356] "Generic (PLEG): container finished" podID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerID="02aebbd376bdd4fd0eedfb0d1f919c92963d89d6ad9185aacfa33a310490c77a" exitCode=137 Sep 16 07:44:50.947181 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:50.947091 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerDied","Data":"e776e1d4d551b2b51c1586bf62e843025eae82f8631ce3d59608d85c75416e4b"} Sep 16 07:44:50.947181 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:50.947137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerDied","Data":"02aebbd376bdd4fd0eedfb0d1f919c92963d89d6ad9185aacfa33a310490c77a"} Sep 16 07:44:50.999190 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:50.999169 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:44:51.108875 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.108850 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/48ce88e1-39e7-4265-bc28-be77b2721f46-feast-data\") pod \"48ce88e1-39e7-4265-bc28-be77b2721f46\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " Sep 16 07:44:51.109040 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.108897 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b56x4\" (UniqueName: \"kubernetes.io/projected/48ce88e1-39e7-4265-bc28-be77b2721f46-kube-api-access-b56x4\") pod \"48ce88e1-39e7-4265-bc28-be77b2721f46\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " Sep 16 07:44:51.109040 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.108917 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-offline-tls\") pod \"48ce88e1-39e7-4265-bc28-be77b2721f46\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " Sep 16 07:44:51.109040 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.108941 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/48ce88e1-39e7-4265-bc28-be77b2721f46-registry-tls\") pod \"48ce88e1-39e7-4265-bc28-be77b2721f46\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " Sep 16 07:44:51.109040 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.109030 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-ui-tls\") pod \"48ce88e1-39e7-4265-bc28-be77b2721f46\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " Sep 16 07:44:51.109252 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.109054 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-online-tls\") pod \"48ce88e1-39e7-4265-bc28-be77b2721f46\" (UID: \"48ce88e1-39e7-4265-bc28-be77b2721f46\") " Sep 16 07:44:51.109524 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.109486 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48ce88e1-39e7-4265-bc28-be77b2721f46-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "48ce88e1-39e7-4265-bc28-be77b2721f46" (UID: "48ce88e1-39e7-4265-bc28-be77b2721f46"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 07:44:51.109617 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.109555 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ce88e1-39e7-4265-bc28-be77b2721f46-feast-data" (OuterVolumeSpecName: "feast-data") pod "48ce88e1-39e7-4265-bc28-be77b2721f46" (UID: "48ce88e1-39e7-4265-bc28-be77b2721f46"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:44:51.111265 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.111221 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48ce88e1-39e7-4265-bc28-be77b2721f46-kube-api-access-b56x4" (OuterVolumeSpecName: "kube-api-access-b56x4") pod "48ce88e1-39e7-4265-bc28-be77b2721f46" (UID: "48ce88e1-39e7-4265-bc28-be77b2721f46"). InnerVolumeSpecName "kube-api-access-b56x4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:44:51.111347 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.111303 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "48ce88e1-39e7-4265-bc28-be77b2721f46" (UID: "48ce88e1-39e7-4265-bc28-be77b2721f46"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:44:51.111571 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.111553 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-online-tls" (OuterVolumeSpecName: "online-tls") pod "48ce88e1-39e7-4265-bc28-be77b2721f46" (UID: "48ce88e1-39e7-4265-bc28-be77b2721f46"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:44:51.111655 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.111575 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "48ce88e1-39e7-4265-bc28-be77b2721f46" (UID: "48ce88e1-39e7-4265-bc28-be77b2721f46"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:44:51.154115 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.154097 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:44:51.209758 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.209738 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-feast-data\") pod \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " Sep 16 07:44:51.209886 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.209804 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-offline-tls\") pod \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " Sep 16 07:44:51.209886 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.209823 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-online-tls\") pod \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " Sep 16 07:44:51.209886 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.209843 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-ui-tls\") pod \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " Sep 16 07:44:51.209886 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.209860 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-registry-tls\") pod \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " Sep 16 07:44:51.210100 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.209894 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pt9c\" (UniqueName: \"kubernetes.io/projected/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-kube-api-access-2pt9c\") pod \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\" (UID: \"a0eb47ab-61c0-41fe-9f4e-70ebed867c36\") " Sep 16 07:44:51.210100 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.210077 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b56x4\" (UniqueName: \"kubernetes.io/projected/48ce88e1-39e7-4265-bc28-be77b2721f46-kube-api-access-b56x4\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.210100 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.210098 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-offline-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.210260 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.210111 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/48ce88e1-39e7-4265-bc28-be77b2721f46-registry-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.210260 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.210124 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-ui-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.210260 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.210137 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/48ce88e1-39e7-4265-bc28-be77b2721f46-online-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.210260 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.210149 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/48ce88e1-39e7-4265-bc28-be77b2721f46-feast-data\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.210460 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.210253 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-feast-data" (OuterVolumeSpecName: "feast-data") pod "a0eb47ab-61c0-41fe-9f4e-70ebed867c36" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:44:51.211797 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.211759 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a0eb47ab-61c0-41fe-9f4e-70ebed867c36" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:44:51.211883 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.211857 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a0eb47ab-61c0-41fe-9f4e-70ebed867c36" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:44:51.212047 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.212015 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-kube-api-access-2pt9c" (OuterVolumeSpecName: "kube-api-access-2pt9c") pod "a0eb47ab-61c0-41fe-9f4e-70ebed867c36" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36"). InnerVolumeSpecName "kube-api-access-2pt9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:44:51.212310 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.212291 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-online-tls" (OuterVolumeSpecName: "online-tls") pod "a0eb47ab-61c0-41fe-9f4e-70ebed867c36" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:44:51.212310 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.212303 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a0eb47ab-61c0-41fe-9f4e-70ebed867c36" (UID: "a0eb47ab-61c0-41fe-9f4e-70ebed867c36"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:44:51.310571 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.310548 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-registry-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.310571 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.310569 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pt9c\" (UniqueName: \"kubernetes.io/projected/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-kube-api-access-2pt9c\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.310683 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.310579 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-feast-data\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.310683 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.310588 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-offline-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.310683 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.310596 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-online-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.310683 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.310603 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0eb47ab-61c0-41fe-9f4e-70ebed867c36-ui-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:44:51.953704 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.953676 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" Sep 16 07:44:51.953704 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.953696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr" event={"ID":"a0eb47ab-61c0-41fe-9f4e-70ebed867c36","Type":"ContainerDied","Data":"3b544cbcff6a6cc0da2f85a6fe629ef35ef890c3d5e2fa45503505f4d8459097"} Sep 16 07:44:51.954167 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.953747 2580 scope.go:117] "RemoveContainer" containerID="5d86371c94294cf41cb677e8dc5819cf07955c142c3e3a0da6acd6a09a8adca6" Sep 16 07:44:51.956799 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.956761 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" event={"ID":"48ce88e1-39e7-4265-bc28-be77b2721f46","Type":"ContainerDied","Data":"94bd1b6bb215c1eab2a5cfae75cd6bdbdd5361d03cb0b90e018aa7200e21b794"} Sep 16 07:44:51.956909 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.956836 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d" Sep 16 07:44:51.963041 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.963005 2580 scope.go:117] "RemoveContainer" containerID="e776e1d4d551b2b51c1586bf62e843025eae82f8631ce3d59608d85c75416e4b" Sep 16 07:44:51.970901 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.970881 2580 scope.go:117] "RemoveContainer" containerID="9a751b7db15cf9ba3bfa3e20c5ef603a6728a169b30b9f755267371d018cf2b5" Sep 16 07:44:51.976894 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.976873 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr"] Sep 16 07:44:51.978321 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.978304 2580 scope.go:117] "RemoveContainer" containerID="02aebbd376bdd4fd0eedfb0d1f919c92963d89d6ad9185aacfa33a310490c77a" Sep 16 07:44:51.980401 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.980380 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b8d6dbb67-9h2pr"] Sep 16 07:44:51.985512 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.985498 2580 scope.go:117] "RemoveContainer" containerID="9fc9080f0c2141f48a100dc667aa81e39cd1a3afb8d3af2a4157dddeffe6130b" Sep 16 07:44:51.989849 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.989828 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d"] Sep 16 07:44:51.995311 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.995295 2580 scope.go:117] "RemoveContainer" containerID="87a4316f40369339e743e45900b2e8060d8341eb8351bd3acbd94b33a6e9c19d" Sep 16 07:44:51.995429 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:51.995410 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7679d98b45-hrd5d"] Sep 16 07:44:52.002012 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:52.001994 2580 scope.go:117] "RemoveContainer" containerID="66754faed7532d585ac877532dd1190763355f8b7be9f8b1340f97d7ee0f4679" Sep 16 07:44:52.008718 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:52.008703 2580 scope.go:117] "RemoveContainer" containerID="91a21c0516f1eaf855305b95bd66fcb5911bbb64a4b8782544591bfa4210ce64" Sep 16 07:44:52.015441 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:52.015426 2580 scope.go:117] "RemoveContainer" containerID="3b04b2caf3dade44978aad97d85b5b054046fa0810663c3d625920fd31071ca5" Sep 16 07:44:52.022139 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:52.022120 2580 scope.go:117] "RemoveContainer" containerID="20f93aa60720a783ef0456365acfb240f21c39dfca5ddcd432f2290ba381f4fc" Sep 16 07:44:52.028667 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:52.028649 2580 scope.go:117] "RemoveContainer" containerID="a0683226abe602762a76bd0acfda3542f3c66904c31a8f991faf05b00e233bd6" Sep 16 07:44:53.059392 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:53.059361 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" path="/var/lib/kubelet/pods/48ce88e1-39e7-4265-bc28-be77b2721f46/volumes" Sep 16 07:44:53.059907 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:44:53.059893 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" path="/var/lib/kubelet/pods/a0eb47ab-61c0-41fe-9f4e-70ebed867c36/volumes" Sep 16 07:45:03.085871 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.085823 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:45:03.277582 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.277541 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-b2zs8"] Sep 16 07:45:03.278017 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.277999 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="feast-apply" Sep 16 07:45:03.278077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278022 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="feast-apply" Sep 16 07:45:03.278077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278034 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="offline" Sep 16 07:45:03.278077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278044 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="offline" Sep 16 07:45:03.278077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278057 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="offline" Sep 16 07:45:03.278077 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278066 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="offline" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278080 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="feast-init" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278089 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="feast-init" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278097 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="registry" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278106 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="registry" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278115 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="ui" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278124 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="ui" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278140 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="online" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278149 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="online" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278166 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="feast-apply" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278174 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="feast-apply" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278189 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="ui" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278197 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="ui" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278210 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="feast-init" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278218 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="feast-init" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278231 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="online" Sep 16 07:45:03.278281 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278239 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="online" Sep 16 07:45:03.278827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278303 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="offline" Sep 16 07:45:03.278827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278317 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="online" Sep 16 07:45:03.278827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278325 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="ui" Sep 16 07:45:03.278827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278333 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="ui" Sep 16 07:45:03.278827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278343 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="online" Sep 16 07:45:03.278827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278352 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48ce88e1-39e7-4265-bc28-be77b2721f46" containerName="offline" Sep 16 07:45:03.278827 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.278360 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0eb47ab-61c0-41fe-9f4e-70ebed867c36" containerName="registry" Sep 16 07:45:03.283454 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.283437 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:45:03.285610 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.285584 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-b2zs8"] Sep 16 07:45:03.285610 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.285605 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 16 07:45:03.286341 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.286315 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 16 07:45:03.286434 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.286360 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 16 07:45:03.286434 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.286380 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-6d2fj\"" Sep 16 07:45:03.312231 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.312205 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:45:03.402269 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.402235 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/857a71a5-cfcf-4c04-a39a-480192a5b911-postgresdata\") pod \"postgres-864846d4f5-b2zs8\" (UID: \"857a71a5-cfcf-4c04-a39a-480192a5b911\") " pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:45:03.402405 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.402292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf7dj\" (UniqueName: \"kubernetes.io/projected/857a71a5-cfcf-4c04-a39a-480192a5b911-kube-api-access-zf7dj\") pod \"postgres-864846d4f5-b2zs8\" (UID: \"857a71a5-cfcf-4c04-a39a-480192a5b911\") " pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:45:03.409399 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.409374 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:45:03.502718 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.502695 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf7dj\" (UniqueName: \"kubernetes.io/projected/857a71a5-cfcf-4c04-a39a-480192a5b911-kube-api-access-zf7dj\") pod \"postgres-864846d4f5-b2zs8\" (UID: \"857a71a5-cfcf-4c04-a39a-480192a5b911\") " pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:45:03.502828 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.502743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/857a71a5-cfcf-4c04-a39a-480192a5b911-postgresdata\") pod \"postgres-864846d4f5-b2zs8\" (UID: \"857a71a5-cfcf-4c04-a39a-480192a5b911\") " pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:45:03.503079 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.503064 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/857a71a5-cfcf-4c04-a39a-480192a5b911-postgresdata\") pod \"postgres-864846d4f5-b2zs8\" (UID: \"857a71a5-cfcf-4c04-a39a-480192a5b911\") " pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:45:03.510542 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.510512 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf7dj\" (UniqueName: \"kubernetes.io/projected/857a71a5-cfcf-4c04-a39a-480192a5b911-kube-api-access-zf7dj\") pod \"postgres-864846d4f5-b2zs8\" (UID: \"857a71a5-cfcf-4c04-a39a-480192a5b911\") " pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:45:03.600511 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.600486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:45:03.867604 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.867507 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-b2zs8"] Sep 16 07:45:03.870276 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:45:03.870239 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod857a71a5_cfcf_4c04_a39a_480192a5b911.slice/crio-b22d2c00dd76d047fa2752750f4e423f0bb69829ba0f7259bf4a331380331d62 WatchSource:0}: Error finding container b22d2c00dd76d047fa2752750f4e423f0bb69829ba0f7259bf4a331380331d62: Status 404 returned error can't find the container with id b22d2c00dd76d047fa2752750f4e423f0bb69829ba0f7259bf4a331380331d62 Sep 16 07:45:03.872011 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:03.871993 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 16 07:45:04.008126 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:04.008075 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-b2zs8" event={"ID":"857a71a5-cfcf-4c04-a39a-480192a5b911","Type":"ContainerStarted","Data":"b22d2c00dd76d047fa2752750f4e423f0bb69829ba0f7259bf4a331380331d62"} Sep 16 07:45:09.029888 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:09.029854 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-b2zs8" event={"ID":"857a71a5-cfcf-4c04-a39a-480192a5b911","Type":"ContainerStarted","Data":"c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df"} Sep 16 07:45:09.044048 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:09.044001 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-b2zs8" podStartSLOduration=1.400573533 podStartE2EDuration="6.043983856s" totalImagesPullingTime="4.643410323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:45:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:45:09.042966476 +0000 UTC m=+1040.567320253" watchObservedRunningTime="2026-09-16 07:45:09.043983856 +0000 UTC m=+1040.568337635" Sep 16 07:45:11.994682 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:11.994620 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:45:12.080585 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.080547 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:45:12.086896 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.086857 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8"] Sep 16 07:45:12.091026 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.091008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.093122 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.093102 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 16 07:45:12.093234 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.093217 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 16 07:45:12.093292 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.093244 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 16 07:45:12.093470 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.093453 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 16 07:45:12.093531 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.093460 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-lnjzc\"" Sep 16 07:45:12.101891 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.101869 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8"] Sep 16 07:45:12.200530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.200495 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:45:12.273060 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.272993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6qk9\" (UniqueName: \"kubernetes.io/projected/028908f6-2622-4a2f-a3ea-90daf8ba3fee-kube-api-access-r6qk9\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.273060 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.273026 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-registry-tls\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.273060 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.273058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-offline-tls\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.273233 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.273135 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028908f6-2622-4a2f-a3ea-90daf8ba3fee-feast-data\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.273233 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.273166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-online-tls\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.301505 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.301482 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:45:12.374361 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.374339 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028908f6-2622-4a2f-a3ea-90daf8ba3fee-feast-data\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.374468 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.374371 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-online-tls\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.374468 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.374402 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6qk9\" (UniqueName: \"kubernetes.io/projected/028908f6-2622-4a2f-a3ea-90daf8ba3fee-kube-api-access-r6qk9\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.374468 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.374421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-registry-tls\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.374642 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.374614 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-offline-tls\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.374705 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.374692 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028908f6-2622-4a2f-a3ea-90daf8ba3fee-feast-data\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.376876 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.376856 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-online-tls\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.376957 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.376873 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-registry-tls\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.376957 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.376925 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-offline-tls\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.382211 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.382192 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6qk9\" (UniqueName: \"kubernetes.io/projected/028908f6-2622-4a2f-a3ea-90daf8ba3fee-kube-api-access-r6qk9\") pod \"feast-credit-scoring-58f9977f4d-t5wk8\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.400849 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.400827 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:45:12.402270 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.402255 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:12.676544 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:12.676486 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8"] Sep 16 07:45:12.678983 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:45:12.678958 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod028908f6_2622_4a2f_a3ea_90daf8ba3fee.slice/crio-31e2f2a61c925138e390465285de69b154145b39eaa2c00316359404c0966e51 WatchSource:0}: Error finding container 31e2f2a61c925138e390465285de69b154145b39eaa2c00316359404c0966e51: Status 404 returned error can't find the container with id 31e2f2a61c925138e390465285de69b154145b39eaa2c00316359404c0966e51 Sep 16 07:45:13.047302 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:13.047222 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerStarted","Data":"f846bf39612109e62b7a6b4efaac9c89d76fc44565eb68bdcca3c167e5d67ce1"} Sep 16 07:45:13.047302 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:13.047292 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerStarted","Data":"31e2f2a61c925138e390465285de69b154145b39eaa2c00316359404c0966e51"} Sep 16 07:45:14.052534 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:14.052474 2580 generic.go:356] "Generic (PLEG): container finished" podID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerID="f846bf39612109e62b7a6b4efaac9c89d76fc44565eb68bdcca3c167e5d67ce1" exitCode=0 Sep 16 07:45:14.052856 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:14.052559 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerDied","Data":"f846bf39612109e62b7a6b4efaac9c89d76fc44565eb68bdcca3c167e5d67ce1"} Sep 16 07:45:15.060208 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:15.060172 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerStarted","Data":"8c8a40ab21cdc6069aa1bb4c6ee52009037937f1b64e756ccd73c223b9ae5239"} Sep 16 07:45:21.083973 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:21.083938 2580 generic.go:356] "Generic (PLEG): container finished" podID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerID="8c8a40ab21cdc6069aa1bb4c6ee52009037937f1b64e756ccd73c223b9ae5239" exitCode=0 Sep 16 07:45:21.084360 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:21.084010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerDied","Data":"8c8a40ab21cdc6069aa1bb4c6ee52009037937f1b64e756ccd73c223b9ae5239"} Sep 16 07:45:22.092564 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:22.092524 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerStarted","Data":"e41d765303099f54734bbd67c4784d2cd584258135b09781687192044fadbe5b"} Sep 16 07:45:22.093022 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:22.092637 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerStarted","Data":"37ab48d2f7f693157c9141fe4b46d3e5ebdde23fab33542d8ef6dbbef4f5ab61"} Sep 16 07:45:22.093022 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:22.092668 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerStarted","Data":"c6e5f6a2e8075006c10984edd92501cb7a3306aae61549dd5123f4f405cc4cd4"} Sep 16 07:45:22.116316 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:22.116257 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podStartSLOduration=10.116237242 podStartE2EDuration="10.116237242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:45:22.114886214 +0000 UTC m=+1053.639240022" watchObservedRunningTime="2026-09-16 07:45:22.116237242 +0000 UTC m=+1053.640591022" Sep 16 07:45:22.403395 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:22.403356 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:22.403555 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:22.403479 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:22.403555 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:22.403508 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:24.403149 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:24.403032 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:24.403149 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:24.403106 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:24.403149 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:24.403137 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:24.404893 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:24.404853 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 16 07:45:24.405018 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:24.404901 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 16 07:45:24.405018 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:24.404900 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 16 07:45:27.403416 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:27.403362 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 16 07:45:27.404005 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:27.403358 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 16 07:45:27.404005 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:27.403382 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 16 07:45:30.403033 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:30.402988 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 16 07:45:30.403591 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:30.403564 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:30.404544 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:30.404521 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:30.409523 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:30.409499 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:31.145840 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:31.145798 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:33.403615 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:33.403573 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 16 07:45:36.403746 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:36.403717 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:36.404420 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:36.404402 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:45:36.775621 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:45:36.775549 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:46:35.336234 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:35.336202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:46:40.317550 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.317514 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8"] Sep 16 07:46:40.318003 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.317818 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="registry" containerID="cri-o://c6e5f6a2e8075006c10984edd92501cb7a3306aae61549dd5123f4f405cc4cd4" gracePeriod=30 Sep 16 07:46:40.318003 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.317826 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="online" containerID="cri-o://37ab48d2f7f693157c9141fe4b46d3e5ebdde23fab33542d8ef6dbbef4f5ab61" gracePeriod=30 Sep 16 07:46:40.318003 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.317825 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="offline" containerID="cri-o://e41d765303099f54734bbd67c4784d2cd584258135b09781687192044fadbe5b" gracePeriod=30 Sep 16 07:46:40.335416 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.335391 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-b2zs8"] Sep 16 07:46:40.335560 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.335543 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-b2zs8" podUID="857a71a5-cfcf-4c04-a39a-480192a5b911" containerName="postgres" containerID="cri-o://c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df" gracePeriod=30 Sep 16 07:46:40.428658 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.428627 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:46:40.573817 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.573744 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:46:40.693252 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.693215 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/857a71a5-cfcf-4c04-a39a-480192a5b911-postgresdata\") pod \"857a71a5-cfcf-4c04-a39a-480192a5b911\" (UID: \"857a71a5-cfcf-4c04-a39a-480192a5b911\") " Sep 16 07:46:40.693494 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.693322 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zf7dj\" (UniqueName: \"kubernetes.io/projected/857a71a5-cfcf-4c04-a39a-480192a5b911-kube-api-access-zf7dj\") pod \"857a71a5-cfcf-4c04-a39a-480192a5b911\" (UID: \"857a71a5-cfcf-4c04-a39a-480192a5b911\") " Sep 16 07:46:40.695374 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.695347 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857a71a5-cfcf-4c04-a39a-480192a5b911-kube-api-access-zf7dj" (OuterVolumeSpecName: "kube-api-access-zf7dj") pod "857a71a5-cfcf-4c04-a39a-480192a5b911" (UID: "857a71a5-cfcf-4c04-a39a-480192a5b911"). InnerVolumeSpecName "kube-api-access-zf7dj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:46:40.723131 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.723106 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857a71a5-cfcf-4c04-a39a-480192a5b911-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "857a71a5-cfcf-4c04-a39a-480192a5b911" (UID: "857a71a5-cfcf-4c04-a39a-480192a5b911"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:46:40.794768 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.794733 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf7dj\" (UniqueName: \"kubernetes.io/projected/857a71a5-cfcf-4c04-a39a-480192a5b911-kube-api-access-zf7dj\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:46:40.794768 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:40.794763 2580 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/857a71a5-cfcf-4c04-a39a-480192a5b911-postgresdata\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:46:41.143369 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.143336 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 16 07:46:41.424999 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.424970 2580 generic.go:356] "Generic (PLEG): container finished" podID="857a71a5-cfcf-4c04-a39a-480192a5b911" containerID="c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df" exitCode=0 Sep 16 07:46:41.425374 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.425013 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-b2zs8" Sep 16 07:46:41.425374 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.425050 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-b2zs8" event={"ID":"857a71a5-cfcf-4c04-a39a-480192a5b911","Type":"ContainerDied","Data":"c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df"} Sep 16 07:46:41.425374 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.425091 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-b2zs8" event={"ID":"857a71a5-cfcf-4c04-a39a-480192a5b911","Type":"ContainerDied","Data":"b22d2c00dd76d047fa2752750f4e423f0bb69829ba0f7259bf4a331380331d62"} Sep 16 07:46:41.425374 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.425111 2580 scope.go:117] "RemoveContainer" containerID="c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df" Sep 16 07:46:41.428437 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.428417 2580 generic.go:356] "Generic (PLEG): container finished" podID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerID="37ab48d2f7f693157c9141fe4b46d3e5ebdde23fab33542d8ef6dbbef4f5ab61" exitCode=0 Sep 16 07:46:41.428540 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.428477 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerDied","Data":"37ab48d2f7f693157c9141fe4b46d3e5ebdde23fab33542d8ef6dbbef4f5ab61"} Sep 16 07:46:41.433051 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.433033 2580 scope.go:117] "RemoveContainer" containerID="c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df" Sep 16 07:46:41.433309 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:46:41.433291 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df\": container with ID starting with c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df not found: ID does not exist" containerID="c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df" Sep 16 07:46:41.433376 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.433318 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df"} err="failed to get container status \"c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df\": rpc error: code = NotFound desc = could not find container \"c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df\": container with ID starting with c95184f849ea1a2135bdc82410efe7a57f29a8197abb58571553d909fe5b23df not found: ID does not exist" Sep 16 07:46:41.451826 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.451800 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-b2zs8"] Sep 16 07:46:41.464081 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:41.464057 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-b2zs8"] Sep 16 07:46:43.060027 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:43.059988 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="857a71a5-cfcf-4c04-a39a-480192a5b911" path="/var/lib/kubelet/pods/857a71a5-cfcf-4c04-a39a-480192a5b911/volumes" Sep 16 07:46:51.143151 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:46:51.143116 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 16 07:47:01.143373 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:01.143334 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 16 07:47:01.143857 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:01.143443 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:47:10.404237 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:10.404205 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 16 07:47:10.538619 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:10.538595 2580 generic.go:356] "Generic (PLEG): container finished" podID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerID="e41d765303099f54734bbd67c4784d2cd584258135b09781687192044fadbe5b" exitCode=137 Sep 16 07:47:10.538619 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:10.538617 2580 generic.go:356] "Generic (PLEG): container finished" podID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerID="c6e5f6a2e8075006c10984edd92501cb7a3306aae61549dd5123f4f405cc4cd4" exitCode=137 Sep 16 07:47:10.538759 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:10.538663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerDied","Data":"e41d765303099f54734bbd67c4784d2cd584258135b09781687192044fadbe5b"} Sep 16 07:47:10.538759 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:10.538717 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerDied","Data":"c6e5f6a2e8075006c10984edd92501cb7a3306aae61549dd5123f4f405cc4cd4"} Sep 16 07:47:10.965344 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:10.965321 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:47:11.115695 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.115674 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-online-tls\") pod \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " Sep 16 07:47:11.115837 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.115710 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-offline-tls\") pod \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " Sep 16 07:47:11.115837 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.115742 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-registry-tls\") pod \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " Sep 16 07:47:11.115837 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.115768 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028908f6-2622-4a2f-a3ea-90daf8ba3fee-feast-data\") pod \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " Sep 16 07:47:11.115837 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.115826 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6qk9\" (UniqueName: \"kubernetes.io/projected/028908f6-2622-4a2f-a3ea-90daf8ba3fee-kube-api-access-r6qk9\") pod \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\" (UID: \"028908f6-2622-4a2f-a3ea-90daf8ba3fee\") " Sep 16 07:47:11.118326 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.118299 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "028908f6-2622-4a2f-a3ea-90daf8ba3fee" (UID: "028908f6-2622-4a2f-a3ea-90daf8ba3fee"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:47:11.118326 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.118315 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-online-tls" (OuterVolumeSpecName: "online-tls") pod "028908f6-2622-4a2f-a3ea-90daf8ba3fee" (UID: "028908f6-2622-4a2f-a3ea-90daf8ba3fee"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:47:11.118498 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.118442 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "028908f6-2622-4a2f-a3ea-90daf8ba3fee" (UID: "028908f6-2622-4a2f-a3ea-90daf8ba3fee"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:47:11.118498 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.118467 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/028908f6-2622-4a2f-a3ea-90daf8ba3fee-kube-api-access-r6qk9" (OuterVolumeSpecName: "kube-api-access-r6qk9") pod "028908f6-2622-4a2f-a3ea-90daf8ba3fee" (UID: "028908f6-2622-4a2f-a3ea-90daf8ba3fee"). InnerVolumeSpecName "kube-api-access-r6qk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:47:11.123833 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.123762 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/028908f6-2622-4a2f-a3ea-90daf8ba3fee-feast-data" (OuterVolumeSpecName: "feast-data") pod "028908f6-2622-4a2f-a3ea-90daf8ba3fee" (UID: "028908f6-2622-4a2f-a3ea-90daf8ba3fee"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:47:11.217024 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.217004 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-online-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:47:11.217024 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.217022 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-offline-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:47:11.217147 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.217031 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028908f6-2622-4a2f-a3ea-90daf8ba3fee-registry-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:47:11.217147 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.217039 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028908f6-2622-4a2f-a3ea-90daf8ba3fee-feast-data\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:47:11.217147 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.217048 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6qk9\" (UniqueName: \"kubernetes.io/projected/028908f6-2622-4a2f-a3ea-90daf8ba3fee-kube-api-access-r6qk9\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:47:11.544720 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.544649 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" event={"ID":"028908f6-2622-4a2f-a3ea-90daf8ba3fee","Type":"ContainerDied","Data":"31e2f2a61c925138e390465285de69b154145b39eaa2c00316359404c0966e51"} Sep 16 07:47:11.544720 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.544687 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8" Sep 16 07:47:11.544720 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.544708 2580 scope.go:117] "RemoveContainer" containerID="e41d765303099f54734bbd67c4784d2cd584258135b09781687192044fadbe5b" Sep 16 07:47:11.553767 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.553747 2580 scope.go:117] "RemoveContainer" containerID="37ab48d2f7f693157c9141fe4b46d3e5ebdde23fab33542d8ef6dbbef4f5ab61" Sep 16 07:47:11.560857 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.560842 2580 scope.go:117] "RemoveContainer" containerID="c6e5f6a2e8075006c10984edd92501cb7a3306aae61549dd5123f4f405cc4cd4" Sep 16 07:47:11.567862 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.567841 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8"] Sep 16 07:47:11.568149 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.568134 2580 scope.go:117] "RemoveContainer" containerID="8c8a40ab21cdc6069aa1bb4c6ee52009037937f1b64e756ccd73c223b9ae5239" Sep 16 07:47:11.574734 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.574682 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-58f9977f4d-t5wk8"] Sep 16 07:47:11.576040 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:11.576026 2580 scope.go:117] "RemoveContainer" containerID="f846bf39612109e62b7a6b4efaac9c89d76fc44565eb68bdcca3c167e5d67ce1" Sep 16 07:47:13.060291 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:13.060254 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" path="/var/lib/kubelet/pods/028908f6-2622-4a2f-a3ea-90daf8ba3fee/volumes" Sep 16 07:47:17.527407 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:17.527314 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:47:17.628104 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:17.628070 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:47:18.349160 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:18.349131 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:47:18.407442 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:18.407411 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:47:18.510379 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:18.510349 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:47:48.905878 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:48.905850 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:47:48.907866 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:47:48.907845 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:48:17.988658 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:48:17.988624 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:48:42.005194 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:48:42.005110 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:21.448709 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.448672 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:21.536406 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.536377 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:21.590234 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590195 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xsq9j"] Sep 16 07:49:21.590515 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590496 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="feast-apply" Sep 16 07:49:21.590608 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590517 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="feast-apply" Sep 16 07:49:21.590608 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590536 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857a71a5-cfcf-4c04-a39a-480192a5b911" containerName="postgres" Sep 16 07:49:21.590608 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590545 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="857a71a5-cfcf-4c04-a39a-480192a5b911" containerName="postgres" Sep 16 07:49:21.590608 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590560 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="online" Sep 16 07:49:21.590608 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590570 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="online" Sep 16 07:49:21.590608 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590580 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="offline" Sep 16 07:49:21.590608 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590588 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="offline" Sep 16 07:49:21.590608 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590607 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="feast-init" Sep 16 07:49:21.591030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590615 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="feast-init" Sep 16 07:49:21.591030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590630 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="registry" Sep 16 07:49:21.591030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590639 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="registry" Sep 16 07:49:21.591030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590758 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="857a71a5-cfcf-4c04-a39a-480192a5b911" containerName="postgres" Sep 16 07:49:21.591030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590771 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="offline" Sep 16 07:49:21.591030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590800 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="online" Sep 16 07:49:21.591030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.590816 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="028908f6-2622-4a2f-a3ea-90daf8ba3fee" containerName="registry" Sep 16 07:49:21.593537 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.593518 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" Sep 16 07:49:21.596097 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.596071 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 16 07:49:21.596740 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.596721 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 16 07:49:21.596860 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.596802 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-z4r49\"" Sep 16 07:49:21.612530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.612509 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xsq9j"] Sep 16 07:49:21.624395 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.624373 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:21.656562 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.656538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7x2t\" (UniqueName: \"kubernetes.io/projected/d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e-kube-api-access-z7x2t\") pod \"redis-69bbd798b6-xsq9j\" (UID: \"d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e\") " pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" Sep 16 07:49:21.757319 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.757265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7x2t\" (UniqueName: \"kubernetes.io/projected/d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e-kube-api-access-z7x2t\") pod \"redis-69bbd798b6-xsq9j\" (UID: \"d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e\") " pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" Sep 16 07:49:21.765060 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.765037 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7x2t\" (UniqueName: \"kubernetes.io/projected/d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e-kube-api-access-z7x2t\") pod \"redis-69bbd798b6-xsq9j\" (UID: \"d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e\") " pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" Sep 16 07:49:21.915574 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:21.915533 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" Sep 16 07:49:22.186729 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:22.186591 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xsq9j"] Sep 16 07:49:22.188906 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:49:22.188878 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd33b2d83_6d15_49cf_b6b1_7ee017a5dd0e.slice/crio-ea979fea15c7cbf674701217d36ff43cb7e894b2c3153f624864ac3ca05d0c5e WatchSource:0}: Error finding container ea979fea15c7cbf674701217d36ff43cb7e894b2c3153f624864ac3ca05d0c5e: Status 404 returned error can't find the container with id ea979fea15c7cbf674701217d36ff43cb7e894b2c3153f624864ac3ca05d0c5e Sep 16 07:49:23.044476 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:23.044436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" event={"ID":"d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e","Type":"ContainerStarted","Data":"ea979fea15c7cbf674701217d36ff43cb7e894b2c3153f624864ac3ca05d0c5e"} Sep 16 07:49:24.049513 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:24.049428 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" event={"ID":"d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e","Type":"ContainerStarted","Data":"bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a"} Sep 16 07:49:24.067859 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:24.067809 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" podStartSLOduration=1.641888298 podStartE2EDuration="3.067795817s" totalImagesPullingTime="1.425907519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:49:24.065558516 +0000 UTC m=+1295.589912294" watchObservedRunningTime="2026-09-16 07:49:24.067795817 +0000 UTC m=+1295.592149587" Sep 16 07:49:26.990113 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:26.990080 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-lb4k9"] Sep 16 07:49:26.993401 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:26.993381 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:49:26.995409 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:26.995389 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 16 07:49:27.000688 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.000663 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-lb4k9"] Sep 16 07:49:27.023579 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.023554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:27.100953 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.100926 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shgzz\" (UniqueName: \"kubernetes.io/projected/99ebac19-30d7-43cb-8ced-77bd3af038f7-kube-api-access-shgzz\") pod \"postgres-864846d4f5-lb4k9\" (UID: \"99ebac19-30d7-43cb-8ced-77bd3af038f7\") " pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:49:27.101111 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.100967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99ebac19-30d7-43cb-8ced-77bd3af038f7-postgresdata\") pod \"postgres-864846d4f5-lb4k9\" (UID: \"99ebac19-30d7-43cb-8ced-77bd3af038f7\") " pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:49:27.201959 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.201921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shgzz\" (UniqueName: \"kubernetes.io/projected/99ebac19-30d7-43cb-8ced-77bd3af038f7-kube-api-access-shgzz\") pod \"postgres-864846d4f5-lb4k9\" (UID: \"99ebac19-30d7-43cb-8ced-77bd3af038f7\") " pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:49:27.202145 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.201977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99ebac19-30d7-43cb-8ced-77bd3af038f7-postgresdata\") pod \"postgres-864846d4f5-lb4k9\" (UID: \"99ebac19-30d7-43cb-8ced-77bd3af038f7\") " pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:49:27.202455 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.202432 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99ebac19-30d7-43cb-8ced-77bd3af038f7-postgresdata\") pod \"postgres-864846d4f5-lb4k9\" (UID: \"99ebac19-30d7-43cb-8ced-77bd3af038f7\") " pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:49:27.209273 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.209246 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shgzz\" (UniqueName: \"kubernetes.io/projected/99ebac19-30d7-43cb-8ced-77bd3af038f7-kube-api-access-shgzz\") pod \"postgres-864846d4f5-lb4k9\" (UID: \"99ebac19-30d7-43cb-8ced-77bd3af038f7\") " pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:49:27.302908 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.302823 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:49:27.572328 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:27.572254 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-lb4k9"] Sep 16 07:49:27.575186 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:49:27.575158 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99ebac19_30d7_43cb_8ced_77bd3af038f7.slice/crio-7309ad321030e13ead6806f08ed7a841198c1028b2d5d12aac660ea78b4cde45 WatchSource:0}: Error finding container 7309ad321030e13ead6806f08ed7a841198c1028b2d5d12aac660ea78b4cde45: Status 404 returned error can't find the container with id 7309ad321030e13ead6806f08ed7a841198c1028b2d5d12aac660ea78b4cde45 Sep 16 07:49:28.067123 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:28.067088 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" event={"ID":"99ebac19-30d7-43cb-8ced-77bd3af038f7","Type":"ContainerStarted","Data":"a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d"} Sep 16 07:49:28.067123 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:28.067128 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" event={"ID":"99ebac19-30d7-43cb-8ced-77bd3af038f7","Type":"ContainerStarted","Data":"7309ad321030e13ead6806f08ed7a841198c1028b2d5d12aac660ea78b4cde45"} Sep 16 07:49:28.083134 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:28.083084 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" podStartSLOduration=2.083067212 podStartE2EDuration="2.083067212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:49:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:49:28.081649833 +0000 UTC m=+1299.606003613" watchObservedRunningTime="2026-09-16 07:49:28.083067212 +0000 UTC m=+1299.607420988" Sep 16 07:49:32.479745 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.479700 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:32.568005 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.567957 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:32.573276 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.573246 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559"] Sep 16 07:49:32.580111 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.580085 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.582432 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.582409 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 16 07:49:32.582588 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.582414 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 16 07:49:32.582588 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.582465 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-246l4\"" Sep 16 07:49:32.582588 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.582520 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 16 07:49:32.589812 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.589758 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559"] Sep 16 07:49:32.631687 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.631656 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:32.646834 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.646803 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-online-tls\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.646991 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.646854 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-feast-data\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.646991 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.646897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-registry-tls\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.646991 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.646925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h56w4\" (UniqueName: \"kubernetes.io/projected/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-kube-api-access-h56w4\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.747817 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.747724 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-online-tls\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.747817 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.747777 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-feast-data\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.747817 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.747813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-registry-tls\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.748044 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.747847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h56w4\" (UniqueName: \"kubernetes.io/projected/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-kube-api-access-h56w4\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.748044 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:49:32.747895 2580 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Sep 16 07:49:32.748044 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:49:32.747962 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-online-tls podName:ff1a5ca3-4ed1-47bd-bacb-57e021e9282b nodeName:}" failed. No retries permitted until 2026-09-16 07:49:33.247940428 +0000 UTC m=+1304.772294281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-online-tls") pod "feast-credit-scoring-99d4cc7bf-vj559" (UID: "ff1a5ca3-4ed1-47bd-bacb-57e021e9282b") : secret "feast-credit-scoring-online-tls" not found Sep 16 07:49:32.748211 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.748190 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-feast-data\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.750412 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.750389 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-registry-tls\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:32.756473 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:32.756440 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h56w4\" (UniqueName: \"kubernetes.io/projected/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-kube-api-access-h56w4\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:33.252118 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:33.252082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-online-tls\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:33.254454 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:33.254428 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-online-tls\") pod \"feast-credit-scoring-99d4cc7bf-vj559\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:33.289413 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:33.289380 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:33.490812 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:33.490753 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:33.770493 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:33.770365 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559"] Sep 16 07:49:33.773317 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:49:33.773291 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1a5ca3_4ed1_47bd_bacb_57e021e9282b.slice/crio-afe0d3c0a83ee5562f5895e59430c5c5f424069aa1df98d05fba26791b0156dc WatchSource:0}: Error finding container afe0d3c0a83ee5562f5895e59430c5c5f424069aa1df98d05fba26791b0156dc: Status 404 returned error can't find the container with id afe0d3c0a83ee5562f5895e59430c5c5f424069aa1df98d05fba26791b0156dc Sep 16 07:49:34.090900 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:34.090819 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerStarted","Data":"7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421"} Sep 16 07:49:34.090900 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:34.090870 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerStarted","Data":"afe0d3c0a83ee5562f5895e59430c5c5f424069aa1df98d05fba26791b0156dc"} Sep 16 07:49:36.099955 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:36.099925 2580 generic.go:356] "Generic (PLEG): container finished" podID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerID="7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421" exitCode=0 Sep 16 07:49:36.100317 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:36.099986 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerDied","Data":"7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421"} Sep 16 07:49:37.106928 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:37.106890 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerStarted","Data":"c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73"} Sep 16 07:49:43.132688 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:43.132661 2580 generic.go:356] "Generic (PLEG): container finished" podID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerID="c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73" exitCode=0 Sep 16 07:49:43.133020 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:43.132732 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerDied","Data":"c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73"} Sep 16 07:49:44.140079 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:44.140045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerStarted","Data":"e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e"} Sep 16 07:49:44.140559 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:44.140097 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerStarted","Data":"d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7"} Sep 16 07:49:44.162055 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:44.162003 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podStartSLOduration=12.161985416 podStartE2EDuration="12.161985416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:49:44.159441057 +0000 UTC m=+1315.683794835" watchObservedRunningTime="2026-09-16 07:49:44.161985416 +0000 UTC m=+1315.686339196" Sep 16 07:49:45.491093 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:45.491058 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:45.491554 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:45.491124 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:45.492748 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:45.492718 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 16 07:49:45.492748 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:45.492729 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 16 07:49:48.491979 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:48.491935 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 16 07:49:48.492456 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:48.491944 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 16 07:49:51.491653 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:51.491610 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 16 07:49:51.499712 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:51.499687 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:51.500143 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:51.500117 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:51.504168 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:51.504148 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:52.812537 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:52.812475 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:53.491308 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:53.491258 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:54.491232 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:54.491190 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 16 07:49:57.491754 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:57.491728 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:57.492468 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:57.492452 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:49:57.882701 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:57.882669 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:58.299530 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.299434 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:58.376643 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.376604 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:58.387575 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.381291 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m"] Sep 16 07:49:58.389914 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.389894 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.392230 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.392208 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 16 07:49:58.392371 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.392210 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 16 07:49:58.392506 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.392483 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 16 07:49:58.392506 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.392496 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-2cddm\"" Sep 16 07:49:58.396230 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.396207 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m"] Sep 16 07:49:58.455372 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.455345 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-registry-tls\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.455493 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.455403 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-online-tls\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.455561 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.455501 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27n9t\" (UniqueName: \"kubernetes.io/projected/5b5cdeb2-0edb-4d11-861c-1769ec22949a-kube-api-access-27n9t\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.455608 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.455568 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5cdeb2-0edb-4d11-861c-1769ec22949a-feast-data\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.556452 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.556396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-online-tls\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.556452 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.556434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27n9t\" (UniqueName: \"kubernetes.io/projected/5b5cdeb2-0edb-4d11-861c-1769ec22949a-kube-api-access-27n9t\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.556889 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.556471 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5cdeb2-0edb-4d11-861c-1769ec22949a-feast-data\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.556889 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.556494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-registry-tls\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.556889 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:49:58.556526 2580 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Sep 16 07:49:58.556889 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:49:58.556592 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-online-tls podName:5b5cdeb2-0edb-4d11-861c-1769ec22949a nodeName:}" failed. No retries permitted until 2026-09-16 07:49:59.056575328 +0000 UTC m=+1330.580929088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-online-tls") pod "feast-driver-ranking-7fffbcf9fd-xhp6m" (UID: "5b5cdeb2-0edb-4d11-861c-1769ec22949a") : secret "feast-driver-ranking-online-tls" not found Sep 16 07:49:58.557043 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.556924 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5cdeb2-0edb-4d11-861c-1769ec22949a-feast-data\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.558764 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.558745 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-registry-tls\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.566062 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.566041 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27n9t\" (UniqueName: \"kubernetes.io/projected/5b5cdeb2-0edb-4d11-861c-1769ec22949a-kube-api-access-27n9t\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:58.587929 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.587909 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:58.688954 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:58.688931 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:49:59.060831 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:59.060806 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-online-tls\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:59.062945 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:59.062929 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-online-tls\") pod \"feast-driver-ranking-7fffbcf9fd-xhp6m\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:59.301091 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:59.301060 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:49:59.580738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:49:59.580713 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m"] Sep 16 07:49:59.582629 ip-10-0-136-65 kubenswrapper[2580]: W0916 07:49:59.582603 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b5cdeb2_0edb_4d11_861c_1769ec22949a.slice/crio-18ce4f0511273d633f212d1d7a5f9f732f585585433151525a2f62002059d1da WatchSource:0}: Error finding container 18ce4f0511273d633f212d1d7a5f9f732f585585433151525a2f62002059d1da: Status 404 returned error can't find the container with id 18ce4f0511273d633f212d1d7a5f9f732f585585433151525a2f62002059d1da Sep 16 07:50:00.211274 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:00.211237 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerStarted","Data":"17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546"} Sep 16 07:50:00.211468 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:00.211288 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerStarted","Data":"18ce4f0511273d633f212d1d7a5f9f732f585585433151525a2f62002059d1da"} Sep 16 07:50:05.233303 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:05.233266 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerID="17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546" exitCode=0 Sep 16 07:50:05.233700 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:05.233341 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerDied","Data":"17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546"} Sep 16 07:50:06.239602 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:06.239566 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerStarted","Data":"aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca"} Sep 16 07:50:12.267623 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:12.267588 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerID="aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca" exitCode=0 Sep 16 07:50:12.267997 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:12.267636 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerDied","Data":"aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca"} Sep 16 07:50:13.276551 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:13.276516 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerStarted","Data":"e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb"} Sep 16 07:50:13.276970 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:13.276572 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerStarted","Data":"ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779"} Sep 16 07:50:13.296740 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:13.296690 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podStartSLOduration=15.296674852 podStartE2EDuration="15.296674852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 07:49:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 07:50:13.29532319 +0000 UTC m=+1344.819676981" watchObservedRunningTime="2026-09-16 07:50:13.296674852 +0000 UTC m=+1344.821028631" Sep 16 07:50:14.302134 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:14.302100 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:50:14.302709 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:14.302673 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:50:14.303306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:14.303277 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 16 07:50:14.303405 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:14.303305 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 16 07:50:17.302137 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:17.302092 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 16 07:50:17.302607 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:17.302214 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 16 07:50:19.301634 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:19.301579 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:50:19.302147 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:19.301671 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:50:20.301447 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:20.301406 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 16 07:50:20.307749 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:20.307724 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:50:20.314517 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:20.314489 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:50:23.302111 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:23.302058 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 16 07:50:26.302068 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:26.302032 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:50:26.302874 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:50:26.302851 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:51:45.549033 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:45.548946 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:47.062929 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.062896 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:47.548865 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.548830 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk"] Sep 16 07:51:47.549121 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.549096 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" podUID="3d0cec32-8d3e-4b67-ac58-0b3c7201db22" containerName="manager" containerID="cri-o://1c268ae4c0b413bacf39b1b9e9182c8d54683bef0e5b76ef43c2258efa64b286" gracePeriod=10 Sep 16 07:51:47.564167 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.564142 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:47.693413 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.693381 2580 generic.go:356] "Generic (PLEG): container finished" podID="3d0cec32-8d3e-4b67-ac58-0b3c7201db22" containerID="1c268ae4c0b413bacf39b1b9e9182c8d54683bef0e5b76ef43c2258efa64b286" exitCode=0 Sep 16 07:51:47.693540 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.693461 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" event={"ID":"3d0cec32-8d3e-4b67-ac58-0b3c7201db22","Type":"ContainerDied","Data":"1c268ae4c0b413bacf39b1b9e9182c8d54683bef0e5b76ef43c2258efa64b286"} Sep 16 07:51:47.807973 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.807911 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" Sep 16 07:51:47.824954 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.824922 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86lvf\" (UniqueName: \"kubernetes.io/projected/3d0cec32-8d3e-4b67-ac58-0b3c7201db22-kube-api-access-86lvf\") pod \"3d0cec32-8d3e-4b67-ac58-0b3c7201db22\" (UID: \"3d0cec32-8d3e-4b67-ac58-0b3c7201db22\") " Sep 16 07:51:47.826312 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.826281 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:47.829226 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.829170 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d0cec32-8d3e-4b67-ac58-0b3c7201db22-kube-api-access-86lvf" (OuterVolumeSpecName: "kube-api-access-86lvf") pod "3d0cec32-8d3e-4b67-ac58-0b3c7201db22" (UID: "3d0cec32-8d3e-4b67-ac58-0b3c7201db22"). InnerVolumeSpecName "kube-api-access-86lvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:51:47.853815 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.853761 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559"] Sep 16 07:51:47.854189 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.854153 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="registry" containerID="cri-o://d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7" gracePeriod=30 Sep 16 07:51:47.854288 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.854214 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="online" containerID="cri-o://e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e" gracePeriod=30 Sep 16 07:51:47.863096 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.863065 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m"] Sep 16 07:51:47.863340 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.863317 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="registry" containerID="cri-o://ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779" gracePeriod=30 Sep 16 07:51:47.863419 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.863337 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="online" containerID="cri-o://e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb" gracePeriod=30 Sep 16 07:51:47.926424 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.926387 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86lvf\" (UniqueName: \"kubernetes.io/projected/3d0cec32-8d3e-4b67-ac58-0b3c7201db22-kube-api-access-86lvf\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:51:47.929352 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:47.929314 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:48.700002 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:48.699976 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerID="e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb" exitCode=0 Sep 16 07:51:48.700323 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:48.700045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerDied","Data":"e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb"} Sep 16 07:51:48.701388 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:48.701367 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" event={"ID":"3d0cec32-8d3e-4b67-ac58-0b3c7201db22","Type":"ContainerDied","Data":"79790ca687dafd7b4a1a1455705107dce80b3981ad58020f7662f82822746538"} Sep 16 07:51:48.701504 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:48.701397 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk" Sep 16 07:51:48.701504 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:48.701421 2580 scope.go:117] "RemoveContainer" containerID="1c268ae4c0b413bacf39b1b9e9182c8d54683bef0e5b76ef43c2258efa64b286" Sep 16 07:51:48.721921 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:48.721903 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk"] Sep 16 07:51:48.725604 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:48.725576 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f77964bcb-4m6lk"] Sep 16 07:51:49.064630 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:49.064604 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d0cec32-8d3e-4b67-ac58-0b3c7201db22" path="/var/lib/kubelet/pods/3d0cec32-8d3e-4b67-ac58-0b3c7201db22/volumes" Sep 16 07:51:49.707883 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:49.707848 2580 generic.go:356] "Generic (PLEG): container finished" podID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerID="e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e" exitCode=0 Sep 16 07:51:49.708257 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:49.707918 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerDied","Data":"e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e"} Sep 16 07:51:49.762455 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:49.762422 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-lb4k9"] Sep 16 07:51:49.762611 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:49.762581 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" podUID="99ebac19-30d7-43cb-8ced-77bd3af038f7" containerName="postgres" containerID="cri-o://a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d" gracePeriod=30 Sep 16 07:51:49.770630 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:49.767151 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xsq9j"] Sep 16 07:51:49.770630 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:49.767327 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" podUID="d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e" containerName="redis" containerID="cri-o://bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a" gracePeriod=30 Sep 16 07:51:49.823414 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:49.823388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:50.024711 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.024688 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:51:50.027507 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.027489 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" Sep 16 07:51:50.146598 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.146555 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7x2t\" (UniqueName: \"kubernetes.io/projected/d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e-kube-api-access-z7x2t\") pod \"d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e\" (UID: \"d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e\") " Sep 16 07:51:50.146824 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.146606 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shgzz\" (UniqueName: \"kubernetes.io/projected/99ebac19-30d7-43cb-8ced-77bd3af038f7-kube-api-access-shgzz\") pod \"99ebac19-30d7-43cb-8ced-77bd3af038f7\" (UID: \"99ebac19-30d7-43cb-8ced-77bd3af038f7\") " Sep 16 07:51:50.146824 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.146648 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99ebac19-30d7-43cb-8ced-77bd3af038f7-postgresdata\") pod \"99ebac19-30d7-43cb-8ced-77bd3af038f7\" (UID: \"99ebac19-30d7-43cb-8ced-77bd3af038f7\") " Sep 16 07:51:50.148874 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.148842 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e-kube-api-access-z7x2t" (OuterVolumeSpecName: "kube-api-access-z7x2t") pod "d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e" (UID: "d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e"). InnerVolumeSpecName "kube-api-access-z7x2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:51:50.148978 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.148907 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99ebac19-30d7-43cb-8ced-77bd3af038f7-kube-api-access-shgzz" (OuterVolumeSpecName: "kube-api-access-shgzz") pod "99ebac19-30d7-43cb-8ced-77bd3af038f7" (UID: "99ebac19-30d7-43cb-8ced-77bd3af038f7"). InnerVolumeSpecName "kube-api-access-shgzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:51:50.176279 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.176236 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99ebac19-30d7-43cb-8ced-77bd3af038f7-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "99ebac19-30d7-43cb-8ced-77bd3af038f7" (UID: "99ebac19-30d7-43cb-8ced-77bd3af038f7"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:51:50.248260 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.248190 2580 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99ebac19-30d7-43cb-8ced-77bd3af038f7-postgresdata\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:51:50.248260 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.248218 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7x2t\" (UniqueName: \"kubernetes.io/projected/d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e-kube-api-access-z7x2t\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:51:50.248260 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.248230 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shgzz\" (UniqueName: \"kubernetes.io/projected/99ebac19-30d7-43cb-8ced-77bd3af038f7-kube-api-access-shgzz\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:51:50.312382 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.312345 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 16 07:51:50.713132 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.713095 2580 generic.go:356] "Generic (PLEG): container finished" podID="99ebac19-30d7-43cb-8ced-77bd3af038f7" containerID="a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d" exitCode=0 Sep 16 07:51:50.713615 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.713136 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" Sep 16 07:51:50.713615 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.713179 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" event={"ID":"99ebac19-30d7-43cb-8ced-77bd3af038f7","Type":"ContainerDied","Data":"a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d"} Sep 16 07:51:50.713615 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.713221 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-lb4k9" event={"ID":"99ebac19-30d7-43cb-8ced-77bd3af038f7","Type":"ContainerDied","Data":"7309ad321030e13ead6806f08ed7a841198c1028b2d5d12aac660ea78b4cde45"} Sep 16 07:51:50.713615 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.713240 2580 scope.go:117] "RemoveContainer" containerID="a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d" Sep 16 07:51:50.714475 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.714452 2580 generic.go:356] "Generic (PLEG): container finished" podID="d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e" containerID="bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a" exitCode=0 Sep 16 07:51:50.714569 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.714501 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" Sep 16 07:51:50.714569 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.714511 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" event={"ID":"d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e","Type":"ContainerDied","Data":"bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a"} Sep 16 07:51:50.714569 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.714538 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-xsq9j" event={"ID":"d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e","Type":"ContainerDied","Data":"ea979fea15c7cbf674701217d36ff43cb7e894b2c3153f624864ac3ca05d0c5e"} Sep 16 07:51:50.722213 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.722195 2580 scope.go:117] "RemoveContainer" containerID="a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d" Sep 16 07:51:50.722487 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:51:50.722459 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d\": container with ID starting with a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d not found: ID does not exist" containerID="a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d" Sep 16 07:51:50.722559 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.722497 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d"} err="failed to get container status \"a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d\": rpc error: code = NotFound desc = could not find container \"a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d\": container with ID starting with a55db03845b324e07af919bdde9321c0490484de3af87d5eb132bc10e4a62b0d not found: ID does not exist" Sep 16 07:51:50.722559 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.722521 2580 scope.go:117] "RemoveContainer" containerID="bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a" Sep 16 07:51:50.729604 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.729588 2580 scope.go:117] "RemoveContainer" containerID="bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a" Sep 16 07:51:50.729855 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:51:50.729833 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a\": container with ID starting with bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a not found: ID does not exist" containerID="bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a" Sep 16 07:51:50.729945 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.729858 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a"} err="failed to get container status \"bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a\": rpc error: code = NotFound desc = could not find container \"bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a\": container with ID starting with bc54849c3c12a26a5bd2b6166fea27f4b8a821cd9f8f397f06181cd89d8e792a not found: ID does not exist" Sep 16 07:51:50.733918 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.733901 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-lb4k9"] Sep 16 07:51:50.736802 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.736771 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-lb4k9"] Sep 16 07:51:50.745397 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.745375 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xsq9j"] Sep 16 07:51:50.747410 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:50.747390 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xsq9j"] Sep 16 07:51:51.060174 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:51.060093 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99ebac19-30d7-43cb-8ced-77bd3af038f7" path="/var/lib/kubelet/pods/99ebac19-30d7-43cb-8ced-77bd3af038f7/volumes" Sep 16 07:51:51.060419 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:51.060405 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e" path="/var/lib/kubelet/pods/d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e/volumes" Sep 16 07:51:51.501099 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:51.501061 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 16 07:51:52.635879 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:52.635847 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:52.697328 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:52.697299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:58.341686 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:58.341652 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:58.440534 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:58.440495 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:51:58.854715 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:51:58.854676 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:00.311753 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:00.311704 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 16 07:52:01.500622 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:01.500581 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 16 07:52:10.312088 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:10.312044 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 16 07:52:10.312453 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:10.312199 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:52:11.501296 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:11.501249 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 16 07:52:11.501730 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:11.501383 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:52:13.109728 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:13.109692 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:13.210630 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:13.210593 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:13.775612 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:13.775576 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:13.879161 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:13.879112 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:14.176992 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:14.176954 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:14.276933 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:14.276902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:14.547199 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:14.547121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:18.528264 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.528236 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:52:18.530886 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.530867 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:52:18.682574 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.682493 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h56w4\" (UniqueName: \"kubernetes.io/projected/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-kube-api-access-h56w4\") pod \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " Sep 16 07:52:18.682574 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.682533 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-online-tls\") pod \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " Sep 16 07:52:18.682574 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.682571 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-online-tls\") pod \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " Sep 16 07:52:18.682872 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.682601 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-registry-tls\") pod \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " Sep 16 07:52:18.682872 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.682717 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27n9t\" (UniqueName: \"kubernetes.io/projected/5b5cdeb2-0edb-4d11-861c-1769ec22949a-kube-api-access-27n9t\") pod \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " Sep 16 07:52:18.682872 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.682830 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5cdeb2-0edb-4d11-861c-1769ec22949a-feast-data\") pod \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\" (UID: \"5b5cdeb2-0edb-4d11-861c-1769ec22949a\") " Sep 16 07:52:18.682872 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.682866 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-feast-data\") pod \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " Sep 16 07:52:18.683069 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.682900 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-registry-tls\") pod \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\" (UID: \"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b\") " Sep 16 07:52:18.683456 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.683373 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5cdeb2-0edb-4d11-861c-1769ec22949a-feast-data" (OuterVolumeSpecName: "feast-data") pod "5b5cdeb2-0edb-4d11-861c-1769ec22949a" (UID: "5b5cdeb2-0edb-4d11-861c-1769ec22949a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:52:18.685037 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.685008 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-online-tls" (OuterVolumeSpecName: "online-tls") pod "5b5cdeb2-0edb-4d11-861c-1769ec22949a" (UID: "5b5cdeb2-0edb-4d11-861c-1769ec22949a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:52:18.685332 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.685307 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5b5cdeb2-0edb-4d11-861c-1769ec22949a" (UID: "5b5cdeb2-0edb-4d11-861c-1769ec22949a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:52:18.685407 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.685325 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5cdeb2-0edb-4d11-861c-1769ec22949a-kube-api-access-27n9t" (OuterVolumeSpecName: "kube-api-access-27n9t") pod "5b5cdeb2-0edb-4d11-861c-1769ec22949a" (UID: "5b5cdeb2-0edb-4d11-861c-1769ec22949a"). InnerVolumeSpecName "kube-api-access-27n9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:52:18.685407 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.685351 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-online-tls" (OuterVolumeSpecName: "online-tls") pod "ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" (UID: "ff1a5ca3-4ed1-47bd-bacb-57e021e9282b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:52:18.685407 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.685355 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-kube-api-access-h56w4" (OuterVolumeSpecName: "kube-api-access-h56w4") pod "ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" (UID: "ff1a5ca3-4ed1-47bd-bacb-57e021e9282b"). InnerVolumeSpecName "kube-api-access-h56w4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 07:52:18.685566 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.685547 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" (UID: "ff1a5ca3-4ed1-47bd-bacb-57e021e9282b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 07:52:18.692660 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.692629 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-feast-data" (OuterVolumeSpecName: "feast-data") pod "ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" (UID: "ff1a5ca3-4ed1-47bd-bacb-57e021e9282b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 07:52:18.784006 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.783950 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-online-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:52:18.784006 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.783996 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-online-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:52:18.784006 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.784005 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5b5cdeb2-0edb-4d11-861c-1769ec22949a-registry-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:52:18.784006 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.784014 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27n9t\" (UniqueName: \"kubernetes.io/projected/5b5cdeb2-0edb-4d11-861c-1769ec22949a-kube-api-access-27n9t\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:52:18.784006 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.784024 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5cdeb2-0edb-4d11-861c-1769ec22949a-feast-data\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:52:18.784306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.784032 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-feast-data\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:52:18.784306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.784040 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-registry-tls\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:52:18.784306 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.784047 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h56w4\" (UniqueName: \"kubernetes.io/projected/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b-kube-api-access-h56w4\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 07:52:18.835170 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.835132 2580 generic.go:356] "Generic (PLEG): container finished" podID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerID="d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7" exitCode=137 Sep 16 07:52:18.835374 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.835213 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" Sep 16 07:52:18.835374 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.835211 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerDied","Data":"d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7"} Sep 16 07:52:18.835486 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.835386 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559" event={"ID":"ff1a5ca3-4ed1-47bd-bacb-57e021e9282b","Type":"ContainerDied","Data":"afe0d3c0a83ee5562f5895e59430c5c5f424069aa1df98d05fba26791b0156dc"} Sep 16 07:52:18.835486 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.835411 2580 scope.go:117] "RemoveContainer" containerID="e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e" Sep 16 07:52:18.837601 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.837566 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerID="ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779" exitCode=137 Sep 16 07:52:18.837738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.837620 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerDied","Data":"ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779"} Sep 16 07:52:18.837738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.837642 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" Sep 16 07:52:18.837738 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.837663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m" event={"ID":"5b5cdeb2-0edb-4d11-861c-1769ec22949a","Type":"ContainerDied","Data":"18ce4f0511273d633f212d1d7a5f9f732f585585433151525a2f62002059d1da"} Sep 16 07:52:18.845157 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.845139 2580 scope.go:117] "RemoveContainer" containerID="d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7" Sep 16 07:52:18.853777 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.853755 2580 scope.go:117] "RemoveContainer" containerID="c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73" Sep 16 07:52:18.861369 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.861241 2580 scope.go:117] "RemoveContainer" containerID="7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421" Sep 16 07:52:18.864348 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.864324 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m"] Sep 16 07:52:18.868691 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.868669 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7fffbcf9fd-xhp6m"] Sep 16 07:52:18.873974 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.873956 2580 scope.go:117] "RemoveContainer" containerID="e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e" Sep 16 07:52:18.874312 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:52:18.874292 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e\": container with ID starting with e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e not found: ID does not exist" containerID="e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e" Sep 16 07:52:18.874361 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.874321 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e"} err="failed to get container status \"e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e\": rpc error: code = NotFound desc = could not find container \"e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e\": container with ID starting with e2d80a2563bbce1b3010cdeac8454f48396d89f6772900929ee3cfc32f828c9e not found: ID does not exist" Sep 16 07:52:18.874361 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.874340 2580 scope.go:117] "RemoveContainer" containerID="d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7" Sep 16 07:52:18.874594 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:52:18.874577 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7\": container with ID starting with d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7 not found: ID does not exist" containerID="d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7" Sep 16 07:52:18.874634 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.874600 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7"} err="failed to get container status \"d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7\": rpc error: code = NotFound desc = could not find container \"d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7\": container with ID starting with d294d434f25dff09ec2243927f3bd25e55c484fe711eeb0951ccee379db37de7 not found: ID does not exist" Sep 16 07:52:18.874634 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.874617 2580 scope.go:117] "RemoveContainer" containerID="c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73" Sep 16 07:52:18.874870 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:52:18.874854 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73\": container with ID starting with c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73 not found: ID does not exist" containerID="c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73" Sep 16 07:52:18.874931 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.874879 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73"} err="failed to get container status \"c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73\": rpc error: code = NotFound desc = could not find container \"c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73\": container with ID starting with c2ef7ee813b274ad46cdecfef919a82128e14150c3a54d288f0d5f4312ab2b73 not found: ID does not exist" Sep 16 07:52:18.874931 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.874898 2580 scope.go:117] "RemoveContainer" containerID="7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421" Sep 16 07:52:18.875161 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:52:18.875144 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421\": container with ID starting with 7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421 not found: ID does not exist" containerID="7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421" Sep 16 07:52:18.875203 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.875166 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421"} err="failed to get container status \"7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421\": rpc error: code = NotFound desc = could not find container \"7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421\": container with ID starting with 7643367fa91105781ff3a0f8de3d83ba725fadf2fa1ae8f554946b18a71ce421 not found: ID does not exist" Sep 16 07:52:18.875203 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.875180 2580 scope.go:117] "RemoveContainer" containerID="e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb" Sep 16 07:52:18.883301 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.883282 2580 scope.go:117] "RemoveContainer" containerID="ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779" Sep 16 07:52:18.883507 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.883484 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559"] Sep 16 07:52:18.886822 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.886778 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-99d4cc7bf-vj559"] Sep 16 07:52:18.891502 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.891480 2580 scope.go:117] "RemoveContainer" containerID="aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca" Sep 16 07:52:18.898462 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.898444 2580 scope.go:117] "RemoveContainer" containerID="17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546" Sep 16 07:52:18.906100 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.906077 2580 scope.go:117] "RemoveContainer" containerID="e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb" Sep 16 07:52:18.906406 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:52:18.906386 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb\": container with ID starting with e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb not found: ID does not exist" containerID="e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb" Sep 16 07:52:18.906463 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.906416 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb"} err="failed to get container status \"e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb\": rpc error: code = NotFound desc = could not find container \"e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb\": container with ID starting with e058b0b2224385d1245b2770cd4182ad0260c88e864c44520c94736299383fdb not found: ID does not exist" Sep 16 07:52:18.906463 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.906435 2580 scope.go:117] "RemoveContainer" containerID="ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779" Sep 16 07:52:18.906690 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:52:18.906672 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779\": container with ID starting with ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779 not found: ID does not exist" containerID="ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779" Sep 16 07:52:18.906756 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.906698 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779"} err="failed to get container status \"ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779\": rpc error: code = NotFound desc = could not find container \"ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779\": container with ID starting with ddc8e2fc4c8cd6c5563b7e19dca56a265147f498611ed31a15e7c3fd233fe779 not found: ID does not exist" Sep 16 07:52:18.906756 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.906722 2580 scope.go:117] "RemoveContainer" containerID="aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca" Sep 16 07:52:18.906981 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:52:18.906965 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca\": container with ID starting with aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca not found: ID does not exist" containerID="aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca" Sep 16 07:52:18.907045 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.906988 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca"} err="failed to get container status \"aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca\": rpc error: code = NotFound desc = could not find container \"aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca\": container with ID starting with aaa27ada0481e8f196147b0002a9734c206a1856df9d8b6216b41adb014263ca not found: ID does not exist" Sep 16 07:52:18.907045 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.907007 2580 scope.go:117] "RemoveContainer" containerID="17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546" Sep 16 07:52:18.907242 ip-10-0-136-65 kubenswrapper[2580]: E0916 07:52:18.907226 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546\": container with ID starting with 17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546 not found: ID does not exist" containerID="17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546" Sep 16 07:52:18.907279 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:18.907247 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546"} err="failed to get container status \"17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546\": rpc error: code = NotFound desc = could not find container \"17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546\": container with ID starting with 17aad2b87b8b2a2cfb43a5ce9d172553f5810053cccf271929cea5f69dd06546 not found: ID does not exist" Sep 16 07:52:19.059897 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:19.059813 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" path="/var/lib/kubelet/pods/5b5cdeb2-0edb-4d11-861c-1769ec22949a/volumes" Sep 16 07:52:19.060338 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:19.060325 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" path="/var/lib/kubelet/pods/ff1a5ca3-4ed1-47bd-bacb-57e021e9282b/volumes" Sep 16 07:52:27.033659 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:27.033622 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:27.088873 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:27.088838 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:27.669965 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:27.669922 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:52:48.931234 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:48.931202 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:52:48.933527 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:52:48.933499 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:53:54.302974 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:53:54.302934 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:53:55.532320 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:53:55.532283 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:55:19.225137 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:55:19.225095 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:55:19.321258 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:55:19.321210 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:55:19.658632 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:55:19.658601 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:55:19.746952 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:55:19.746915 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:55:19.942679 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:55:19.942586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:55:20.042694 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:55:20.042659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:55:21.011008 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:55:21.010950 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:56:32.931058 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:56:32.931009 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:56:32.988349 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:56:32.988317 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:56:37.361623 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:56:37.361578 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:04.238182 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:04.238140 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:04.290187 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:04.290136 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:04.361933 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:04.361895 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:04.461584 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:04.461548 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:04.562213 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:04.562130 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:09.535165 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:09.535126 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:43.772508 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:43.772425 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:43.884183 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:43.884134 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:48.955938 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:48.955910 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:57:48.959390 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:48.959366 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 07:57:49.455166 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:49.455131 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:57:49.506874 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:57:49.506829 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:58:10.838071 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:58:10.838021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:58:10.893539 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:58:10.893499 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:58:14.141338 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:58:14.141299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:58:14.237434 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:58:14.237397 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:58:14.584825 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:58:14.584696 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:58:14.684695 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:58:14.684660 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:58:14.876101 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:58:14.876072 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:58:15.659030 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:58:15.658992 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:58:23.841878 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:58:23.841842 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:59:39.733370 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:59:39.733330 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 07:59:41.056290 ip-10-0-136-65 kubenswrapper[2580]: I0916 07:59:41.056248 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:01:04.302746 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:01:04.302661 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:01:04.423537 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:01:04.423501 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:01:04.651984 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:01:04.651949 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:01:04.748707 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:01:04.748667 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:01:04.823182 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:01:04.823145 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:01:05.000710 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:01:05.000603 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:01:05.114931 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:01:05.114895 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:02:07.832654 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:02:07.832622 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:02:46.856682 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:02:46.856646 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:02:46.957250 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:02:46.957217 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:02:47.038615 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:02:47.038584 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:02:47.104693 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:02:47.104659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:02:48.983042 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:02:48.983006 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 08:02:48.986683 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:02:48.986653 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 08:02:52.279078 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:02:52.279040 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:02:57.307179 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:02:57.307145 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:29.831458 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:29.831418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:29.965648 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:29.965612 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:35.564177 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:35.564143 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:35.620268 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:35.620225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:36.142467 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142426 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pb8jc/must-gather-nfv7x"] Sep 16 08:03:36.142910 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142889 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="feast-apply" Sep 16 08:03:36.142910 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142911 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="feast-apply" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142926 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99ebac19-30d7-43cb-8ced-77bd3af038f7" containerName="postgres" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142935 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="99ebac19-30d7-43cb-8ced-77bd3af038f7" containerName="postgres" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142946 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="online" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142954 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="online" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142963 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="feast-apply" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142970 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="feast-apply" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142980 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d0cec32-8d3e-4b67-ac58-0b3c7201db22" containerName="manager" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142988 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d0cec32-8d3e-4b67-ac58-0b3c7201db22" containerName="manager" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.142999 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="registry" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143006 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="registry" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143037 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e" containerName="redis" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143045 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e" containerName="redis" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143055 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="feast-init" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143064 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="feast-init" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143075 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="feast-init" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143084 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="feast-init" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143095 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="registry" Sep 16 08:03:36.143095 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143103 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="registry" Sep 16 08:03:36.144016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143112 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="online" Sep 16 08:03:36.144016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143121 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="online" Sep 16 08:03:36.144016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143194 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d0cec32-8d3e-4b67-ac58-0b3c7201db22" containerName="manager" Sep 16 08:03:36.144016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143206 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="registry" Sep 16 08:03:36.144016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143217 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff1a5ca3-4ed1-47bd-bacb-57e021e9282b" containerName="online" Sep 16 08:03:36.144016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143227 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d33b2d83-6d15-49cf-b6b1-7ee017a5dd0e" containerName="redis" Sep 16 08:03:36.144016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143237 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="registry" Sep 16 08:03:36.144016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143244 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="99ebac19-30d7-43cb-8ced-77bd3af038f7" containerName="postgres" Sep 16 08:03:36.144016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.143252 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5cdeb2-0edb-4d11-861c-1769ec22949a" containerName="online" Sep 16 08:03:36.146560 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.146539 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:36.148840 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.148821 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pb8jc\"/\"openshift-service-ca.crt\"" Sep 16 08:03:36.148951 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.148889 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pb8jc\"/\"kube-root-ca.crt\"" Sep 16 08:03:36.149029 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.149014 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pb8jc\"/\"default-dockercfg-8q5tz\"" Sep 16 08:03:36.153779 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.153757 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pb8jc/must-gather-nfv7x"] Sep 16 08:03:36.171762 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.171730 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:36.252227 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.252197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/01284795-6125-41db-84fb-19a44e52c848-must-gather-output\") pod \"must-gather-nfv7x\" (UID: \"01284795-6125-41db-84fb-19a44e52c848\") " pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:36.252227 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.252226 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4j22\" (UniqueName: \"kubernetes.io/projected/01284795-6125-41db-84fb-19a44e52c848-kube-api-access-n4j22\") pod \"must-gather-nfv7x\" (UID: \"01284795-6125-41db-84fb-19a44e52c848\") " pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:36.271938 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.271894 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:36.353272 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.353234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/01284795-6125-41db-84fb-19a44e52c848-must-gather-output\") pod \"must-gather-nfv7x\" (UID: \"01284795-6125-41db-84fb-19a44e52c848\") " pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:36.353272 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.353271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4j22\" (UniqueName: \"kubernetes.io/projected/01284795-6125-41db-84fb-19a44e52c848-kube-api-access-n4j22\") pod \"must-gather-nfv7x\" (UID: \"01284795-6125-41db-84fb-19a44e52c848\") " pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:36.353619 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.353597 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/01284795-6125-41db-84fb-19a44e52c848-must-gather-output\") pod \"must-gather-nfv7x\" (UID: \"01284795-6125-41db-84fb-19a44e52c848\") " pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:36.360829 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.360782 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4j22\" (UniqueName: \"kubernetes.io/projected/01284795-6125-41db-84fb-19a44e52c848-kube-api-access-n4j22\") pod \"must-gather-nfv7x\" (UID: \"01284795-6125-41db-84fb-19a44e52c848\") " pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:36.469382 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.469287 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:36.737091 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.737005 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pb8jc/must-gather-nfv7x"] Sep 16 08:03:36.740146 ip-10-0-136-65 kubenswrapper[2580]: W0916 08:03:36.740121 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01284795_6125_41db_84fb_19a44e52c848.slice/crio-9d0acd9e16d24eee1e4f6147a2c2b5b6fb5fb9acf4a8599b446ce5cc9128e8fa WatchSource:0}: Error finding container 9d0acd9e16d24eee1e4f6147a2c2b5b6fb5fb9acf4a8599b446ce5cc9128e8fa: Status 404 returned error can't find the container with id 9d0acd9e16d24eee1e4f6147a2c2b5b6fb5fb9acf4a8599b446ce5cc9128e8fa Sep 16 08:03:36.742122 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:36.742105 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 16 08:03:37.409124 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:37.409080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" event={"ID":"01284795-6125-41db-84fb-19a44e52c848","Type":"ContainerStarted","Data":"9d0acd9e16d24eee1e4f6147a2c2b5b6fb5fb9acf4a8599b446ce5cc9128e8fa"} Sep 16 08:03:43.438661 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:43.438620 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" event={"ID":"01284795-6125-41db-84fb-19a44e52c848","Type":"ContainerStarted","Data":"cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0"} Sep 16 08:03:43.438661 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:43.438664 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" event={"ID":"01284795-6125-41db-84fb-19a44e52c848","Type":"ContainerStarted","Data":"1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373"} Sep 16 08:03:43.453894 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:43.453844 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" podStartSLOduration=1.139876358 podStartE2EDuration="7.453829569s" totalImagesPullingTime="6.313953211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 08:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 08:03:43.452899626 +0000 UTC m=+2154.977253405" watchObservedRunningTime="2026-09-16 08:03:43.453829569 +0000 UTC m=+2154.978183368" Sep 16 08:03:53.476994 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:53.476959 2580 generic.go:356] "Generic (PLEG): container finished" podID="01284795-6125-41db-84fb-19a44e52c848" containerID="1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373" exitCode=0 Sep 16 08:03:53.477464 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:53.477012 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" event={"ID":"01284795-6125-41db-84fb-19a44e52c848","Type":"ContainerDied","Data":"1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373"} Sep 16 08:03:53.477464 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:53.477305 2580 scope.go:117] "RemoveContainer" containerID="1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373" Sep 16 08:03:53.645018 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:53.644974 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pb8jc_must-gather-nfv7x_01284795-6125-41db-84fb-19a44e52c848/gather/0.log" Sep 16 08:03:53.980346 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:53.980301 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:54.119469 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:54.119423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:54.172420 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:54.172391 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:57.072437 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:57.072409 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-crdsq_cbfcdb7d-38d2-4284-b416-750df96a620a/global-pull-secret-syncer/0.log" Sep 16 08:03:57.217513 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:57.217483 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" Sep 16 08:03:57.288484 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:57.288447 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-65.ec2.internal_98bcfe4ba9a3e44508035277e4f353da/haproxy/0.log" Sep 16 08:03:58.962628 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:58.962584 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-pb8jc/must-gather-nfv7x"] Sep 16 08:03:58.963138 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:58.962850 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" podUID="01284795-6125-41db-84fb-19a44e52c848" containerName="copy" containerID="cri-o://cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0" gracePeriod=2 Sep 16 08:03:58.970488 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:58.970463 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-pb8jc/must-gather-nfv7x"] Sep 16 08:03:58.970820 ip-10-0-136-65 kubenswrapper[2580]: E0916 08:03:58.970769 2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"01284795-6125-41db-84fb-19a44e52c848\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-pb8jc\"/\"must-gather-nfv7x\": pods \"must-gather-nfv7x\" not found" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:59.058374 ip-10-0-136-65 kubenswrapper[2580]: E0916 08:03:59.058338 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nfv7x\" is forbidden: User \"system:node:ip-10-0-136-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-pb8jc\": no relationship found between node 'ip-10-0-136-65.ec2.internal' and this object" podUID="01284795-6125-41db-84fb-19a44e52c848" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:59.189545 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.189524 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pb8jc_must-gather-nfv7x_01284795-6125-41db-84fb-19a44e52c848/copy/0.log" Sep 16 08:03:59.191042 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.191025 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:59.356080 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.356054 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4j22\" (UniqueName: \"kubernetes.io/projected/01284795-6125-41db-84fb-19a44e52c848-kube-api-access-n4j22\") pod \"01284795-6125-41db-84fb-19a44e52c848\" (UID: \"01284795-6125-41db-84fb-19a44e52c848\") " Sep 16 08:03:59.356222 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.356126 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/01284795-6125-41db-84fb-19a44e52c848-must-gather-output\") pod \"01284795-6125-41db-84fb-19a44e52c848\" (UID: \"01284795-6125-41db-84fb-19a44e52c848\") " Sep 16 08:03:59.356468 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.356447 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01284795-6125-41db-84fb-19a44e52c848-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "01284795-6125-41db-84fb-19a44e52c848" (UID: "01284795-6125-41db-84fb-19a44e52c848"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 08:03:59.358251 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.358230 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01284795-6125-41db-84fb-19a44e52c848-kube-api-access-n4j22" (OuterVolumeSpecName: "kube-api-access-n4j22") pod "01284795-6125-41db-84fb-19a44e52c848" (UID: "01284795-6125-41db-84fb-19a44e52c848"). InnerVolumeSpecName "kube-api-access-n4j22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 08:03:59.457453 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.457410 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4j22\" (UniqueName: \"kubernetes.io/projected/01284795-6125-41db-84fb-19a44e52c848-kube-api-access-n4j22\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 08:03:59.457453 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.457447 2580 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/01284795-6125-41db-84fb-19a44e52c848-must-gather-output\") on node \"ip-10-0-136-65.ec2.internal\" DevicePath \"\"" Sep 16 08:03:59.467579 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.467553 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:03:59.500665 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.500643 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pb8jc_must-gather-nfv7x_01284795-6125-41db-84fb-19a44e52c848/copy/0.log" Sep 16 08:03:59.501016 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.500993 2580 generic.go:356] "Generic (PLEG): container finished" podID="01284795-6125-41db-84fb-19a44e52c848" containerID="cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0" exitCode=143 Sep 16 08:03:59.501073 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.501047 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pb8jc/must-gather-nfv7x" Sep 16 08:03:59.501110 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.501070 2580 scope.go:117] "RemoveContainer" containerID="cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0" Sep 16 08:03:59.510654 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.510586 2580 scope.go:117] "RemoveContainer" containerID="1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373" Sep 16 08:03:59.523342 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.523331 2580 scope.go:117] "RemoveContainer" containerID="cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0" Sep 16 08:03:59.523553 ip-10-0-136-65 kubenswrapper[2580]: E0916 08:03:59.523535 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0\": container with ID starting with cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0 not found: ID does not exist" containerID="cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0" Sep 16 08:03:59.523611 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.523561 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0"} err="failed to get container status \"cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0\": rpc error: code = NotFound desc = could not find container \"cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0\": container with ID starting with cfdcfc4f823a12768fe3fc39b95298ccd9bcee9f0a1953c7aea455d933bd0fd0 not found: ID does not exist" Sep 16 08:03:59.523611 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.523583 2580 scope.go:117] "RemoveContainer" containerID="1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373" Sep 16 08:03:59.523840 ip-10-0-136-65 kubenswrapper[2580]: E0916 08:03:59.523821 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373\": container with ID starting with 1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373 not found: ID does not exist" containerID="1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373" Sep 16 08:03:59.523959 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.523844 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373"} err="failed to get container status \"1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373\": rpc error: code = NotFound desc = could not find container \"1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373\": container with ID starting with 1587ff937b9a7414090c52e9ae282443e98933e63ea03aaa774a53828f510373 not found: ID does not exist" Sep 16 08:03:59.527699 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:03:59.527654 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkmk_4b4e05ee-c901-4380-91d5-714df46eec55/konnectivity-agent/0.log" limit=1024 Sep 16 08:04:01.059454 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.059423 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01284795-6125-41db-84fb-19a44e52c848" path="/var/lib/kubelet/pods/01284795-6125-41db-84fb-19a44e52c848/volumes" Sep 16 08:04:01.166320 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.166294 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_611c9bef-8477-4b8b-9d87-67f16e43ced4/alertmanager/0.log" Sep 16 08:04:01.199092 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.199067 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_611c9bef-8477-4b8b-9d87-67f16e43ced4/config-reloader/0.log" Sep 16 08:04:01.220583 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.220560 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_611c9bef-8477-4b8b-9d87-67f16e43ced4/kube-rbac-proxy-web/0.log" Sep 16 08:04:01.244476 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.244440 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_611c9bef-8477-4b8b-9d87-67f16e43ced4/kube-rbac-proxy/0.log" Sep 16 08:04:01.268620 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.268597 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_611c9bef-8477-4b8b-9d87-67f16e43ced4/kube-rbac-proxy-metric/0.log" Sep 16 08:04:01.292982 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.292960 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_611c9bef-8477-4b8b-9d87-67f16e43ced4/prom-label-proxy/0.log" Sep 16 08:04:01.321952 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.321878 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_611c9bef-8477-4b8b-9d87-67f16e43ced4/init-config-reloader/0.log" Sep 16 08:04:01.366540 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.366503 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-24djp_4f2e49ce-6d4b-4ae6-a83c-84a8acc85bdc/cluster-monitoring-operator/0.log" Sep 16 08:04:01.615134 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.615107 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gtqlp_a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25/node-exporter/0.log" Sep 16 08:04:01.634951 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.634929 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gtqlp_a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25/kube-rbac-proxy/0.log" Sep 16 08:04:01.654241 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.654217 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gtqlp_a1aaf3ec-6b99-4e21-b908-e8c8f7a14d25/init-textfile/0.log" Sep 16 08:04:01.754575 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.754540 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-mxs62_666114d6-52da-4dab-b0b6-2ff0c292ab30/kube-rbac-proxy-main/0.log" Sep 16 08:04:01.772075 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.772046 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-mxs62_666114d6-52da-4dab-b0b6-2ff0c292ab30/kube-rbac-proxy-self/0.log" Sep 16 08:04:01.791088 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:01.791054 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-mxs62_666114d6-52da-4dab-b0b6-2ff0c292ab30/openshift-state-metrics/0.log" Sep 16 08:04:02.060640 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:02.060561 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-555c88dcff-x8j6z_84dd45bb-834e-415d-a570-1d065b4d1f09/telemeter-client/0.log" Sep 16 08:04:02.091233 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:02.091207 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-555c88dcff-x8j6z_84dd45bb-834e-415d-a570-1d065b4d1f09/reload/0.log" Sep 16 08:04:02.112395 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:02.112368 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-555c88dcff-x8j6z_84dd45bb-834e-415d-a570-1d065b4d1f09/kube-rbac-proxy/0.log" Sep 16 08:04:03.870528 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:03.870500 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/1.log" Sep 16 08:04:03.878946 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:03.878916 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bhpfr_84595af1-2295-41ac-abe5-c448a3116f98/console-operator/2.log" Sep 16 08:04:04.250699 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.250611 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-tn5cw_cdb9ee11-ed4b-4669-bf34-83514dc6adb3/download-server/0.log" Sep 16 08:04:04.583631 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.583546 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v"] Sep 16 08:04:04.583862 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.583850 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01284795-6125-41db-84fb-19a44e52c848" containerName="copy" Sep 16 08:04:04.583920 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.583863 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="01284795-6125-41db-84fb-19a44e52c848" containerName="copy" Sep 16 08:04:04.583920 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.583875 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01284795-6125-41db-84fb-19a44e52c848" containerName="gather" Sep 16 08:04:04.583920 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.583882 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="01284795-6125-41db-84fb-19a44e52c848" containerName="gather" Sep 16 08:04:04.584034 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.583937 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="01284795-6125-41db-84fb-19a44e52c848" containerName="gather" Sep 16 08:04:04.584034 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.583949 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="01284795-6125-41db-84fb-19a44e52c848" containerName="copy" Sep 16 08:04:04.586156 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.586136 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.588168 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.588147 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n9r4p\"/\"default-dockercfg-n5fgq\"" Sep 16 08:04:04.588287 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.588147 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n9r4p\"/\"openshift-service-ca.crt\"" Sep 16 08:04:04.588776 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.588761 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n9r4p\"/\"kube-root-ca.crt\"" Sep 16 08:04:04.596837 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.596814 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v"] Sep 16 08:04:04.696622 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.696583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-sys\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.696823 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.696626 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-lib-modules\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.696823 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.696710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-proc\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.696823 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.696743 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk5c8\" (UniqueName: \"kubernetes.io/projected/28ed4a8b-dc73-4311-8a7c-6c23052e0807-kube-api-access-mk5c8\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.696823 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.696767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-podres\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.798065 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.798029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-proc\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.798265 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.798075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk5c8\" (UniqueName: \"kubernetes.io/projected/28ed4a8b-dc73-4311-8a7c-6c23052e0807-kube-api-access-mk5c8\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.798265 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.798106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-podres\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.798265 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.798150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-sys\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.798265 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.798162 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-proc\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.798265 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.798187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-lib-modules\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.798265 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.798224 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-sys\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.798470 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.798289 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-podres\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.798470 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.798300 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28ed4a8b-dc73-4311-8a7c-6c23052e0807-lib-modules\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.805992 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.805972 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk5c8\" (UniqueName: \"kubernetes.io/projected/28ed4a8b-dc73-4311-8a7c-6c23052e0807-kube-api-access-mk5c8\") pod \"perf-node-gather-daemonset-9tc8v\" (UID: \"28ed4a8b-dc73-4311-8a7c-6c23052e0807\") " pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:04.896081 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:04.896054 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:05.164225 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:05.164198 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v"] Sep 16 08:04:05.166894 ip-10-0-136-65 kubenswrapper[2580]: W0916 08:04:05.166868 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod28ed4a8b_dc73_4311_8a7c_6c23052e0807.slice/crio-235ade38202eb986e8014d2daf808aa234dfe2fabc4fe2d6776629e4e5962106 WatchSource:0}: Error finding container 235ade38202eb986e8014d2daf808aa234dfe2fabc4fe2d6776629e4e5962106: Status 404 returned error can't find the container with id 235ade38202eb986e8014d2daf808aa234dfe2fabc4fe2d6776629e4e5962106 Sep 16 08:04:05.502871 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:05.502770 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmll6_ddf610bc-54f4-45e1-9c24-dec4ce0048b6/dns/0.log" Sep 16 08:04:05.523441 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:05.523417 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmll6_ddf610bc-54f4-45e1-9c24-dec4ce0048b6/kube-rbac-proxy/0.log" Sep 16 08:04:05.527006 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:05.526982 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" event={"ID":"28ed4a8b-dc73-4311-8a7c-6c23052e0807","Type":"ContainerStarted","Data":"6dd3645837b056bf7040aa1154be5dd0da1047a8510fd2652656bf3dd7466ee0"} Sep 16 08:04:05.527152 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:05.527024 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" event={"ID":"28ed4a8b-dc73-4311-8a7c-6c23052e0807","Type":"ContainerStarted","Data":"235ade38202eb986e8014d2daf808aa234dfe2fabc4fe2d6776629e4e5962106"} Sep 16 08:04:05.527152 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:05.527132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:05.544340 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:05.544296 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" podStartSLOduration=1.54428142 podStartE2EDuration="1.54428142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 08:04:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 08:04:05.542377873 +0000 UTC m=+2177.066731652" watchObservedRunningTime="2026-09-16 08:04:05.54428142 +0000 UTC m=+2177.068635199" Sep 16 08:04:05.627698 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:05.627674 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hjlx2_92f584a7-6d57-4f6b-9f18-b49f43a95e4e/dns-node-resolver/0.log" Sep 16 08:04:05.994038 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:05.994002 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5565947dd6-vvjtz_1f53e1b9-b774-4fb4-89c8-a4bcf604e295/registry/0.log" Sep 16 08:04:06.013883 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:06.013856 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-962g2_28a8c6e2-ffc5-4951-9c7e-5d35d2d34dd2/node-ca/0.log" Sep 16 08:04:07.033002 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:07.032974 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zrfpb_5b0aee94-ad88-4d46-93f1-01ccd630a7cd/serve-healthcheck-canary/0.log" Sep 16 08:04:07.351594 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:07.351513 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2gjlc_2c9e87ea-05bc-40ce-ad08-37a06216c67c/kube-rbac-proxy/0.log" Sep 16 08:04:07.370387 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:07.370359 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2gjlc_2c9e87ea-05bc-40ce-ad08-37a06216c67c/exporter/0.log" Sep 16 08:04:07.390126 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:07.390099 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2gjlc_2c9e87ea-05bc-40ce-ad08-37a06216c67c/extractor/0.log" Sep 16 08:04:11.540460 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:11.540435 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n9r4p/perf-node-gather-daemonset-9tc8v" Sep 16 08:04:12.319548 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.319513 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5mpgc_4c658af4-a137-4b9e-9573-b66f4b836ba9/kube-multus/0.log" Sep 16 08:04:12.508701 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.508677 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mq8vr_6d34d815-6ad3-4107-9499-e004e45cb6ba/kube-multus-additional-cni-plugins/0.log" Sep 16 08:04:12.529623 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.529587 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mq8vr_6d34d815-6ad3-4107-9499-e004e45cb6ba/egress-router-binary-copy/0.log" Sep 16 08:04:12.549636 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.549618 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mq8vr_6d34d815-6ad3-4107-9499-e004e45cb6ba/cni-plugins/0.log" Sep 16 08:04:12.569612 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.569544 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mq8vr_6d34d815-6ad3-4107-9499-e004e45cb6ba/bond-cni-plugin/0.log" Sep 16 08:04:12.589924 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.589903 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mq8vr_6d34d815-6ad3-4107-9499-e004e45cb6ba/routeoverride-cni/0.log" Sep 16 08:04:12.608527 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.608505 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mq8vr_6d34d815-6ad3-4107-9499-e004e45cb6ba/whereabouts-cni-bincopy/0.log" Sep 16 08:04:12.628707 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.628684 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mq8vr_6d34d815-6ad3-4107-9499-e004e45cb6ba/whereabouts-cni/0.log" Sep 16 08:04:12.808712 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.808688 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-99h7c_e1477d2a-2a8c-4b94-b45f-d8d82ece6aab/network-metrics-daemon/0.log" Sep 16 08:04:12.828843 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:12.828767 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-99h7c_e1477d2a-2a8c-4b94-b45f-d8d82ece6aab/kube-rbac-proxy/0.log" Sep 16 08:04:13.906274 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:13.906162 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9rxmw_99e8b563-af63-4f93-b619-8a7e4c07603d/ovn-controller/0.log" Sep 16 08:04:13.923243 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:13.923217 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9rxmw_99e8b563-af63-4f93-b619-8a7e4c07603d/ovn-acl-logging/0.log" Sep 16 08:04:13.950440 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:13.950411 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9rxmw_99e8b563-af63-4f93-b619-8a7e4c07603d/kube-rbac-proxy-node/0.log" Sep 16 08:04:13.974115 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:13.974037 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9rxmw_99e8b563-af63-4f93-b619-8a7e4c07603d/kube-rbac-proxy-ovn-metrics/0.log" Sep 16 08:04:13.989738 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:13.989687 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9rxmw_99e8b563-af63-4f93-b619-8a7e4c07603d/northd/0.log" Sep 16 08:04:14.011306 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:14.011273 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9rxmw_99e8b563-af63-4f93-b619-8a7e4c07603d/nbdb/0.log" Sep 16 08:04:14.038456 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:14.038424 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9rxmw_99e8b563-af63-4f93-b619-8a7e4c07603d/sbdb/0.log" Sep 16 08:04:14.092039 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:14.092010 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9rxmw_99e8b563-af63-4f93-b619-8a7e4c07603d/ovnkube-controller/0.log" Sep 16 08:04:15.274883 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:15.274769 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-6f8b768455-qhcvg_6c1969b6-ef6f-4527-ad67-3dd392558050/check-endpoints/0.log" Sep 16 08:04:15.342476 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:15.342446 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-zcss9_d68eaab1-3586-4f06-8c86-d66a9908733a/network-check-target-container/0.log" Sep 16 08:04:16.153874 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:16.153841 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qdfwd_67eb3291-276f-4393-b1f0-85f4eb5b1887/iptables-alerter/0.log" Sep 16 08:04:16.772509 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:16.772479 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-j6lxc_8535a87b-98d2-487c-9ed7-e9c9d4468e71/tuned/0.log" Sep 16 08:04:18.341916 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:18.341877 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-86499c89c4-tj6dv_b8e9d14a-fed3-497f-aa3d-d7c8e783fa01/cluster-samples-operator/0.log" Sep 16 08:04:18.359480 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:18.359451 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-86499c89c4-tj6dv_b8e9d14a-fed3-497f-aa3d-d7c8e783fa01/cluster-samples-operator-watch/0.log" Sep 16 08:04:19.463676 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:19.463636 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-568869fdc5-shw59_45bcc6e9-fc0c-472d-b84d-ba7dc809f0b5/service-ca-controller/0.log" Sep 16 08:04:19.820733 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:19.820662 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-b24st_06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b/csi-driver/0.log" Sep 16 08:04:19.840778 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:19.840723 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-b24st_06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b/csi-node-driver-registrar/0.log" Sep 16 08:04:19.859326 ip-10-0-136-65 kubenswrapper[2580]: I0916 08:04:19.859297 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-b24st_06cf6a1f-84a9-4967-9ae0-f0dd8d05ec0b/csi-liveness-probe/0.log"