Sep 18 03:15:53.175040 ip-10-0-129-90 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 18 03:15:53.175050 ip-10-0-129-90 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 18 03:15:53.175057 ip-10-0-129-90 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 18 03:15:53.175267 ip-10-0-129-90 systemd[1]: Failed to start Kubernetes Kubelet. Sep 18 03:16:04.331054 ip-10-0-129-90 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 18 03:16:04.331070 ip-10-0-129-90 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 409e560aa00443559e244ef5f52a645a -- Sep 18 03:17:40.673422 ip-10-0-129-90 systemd[1]: Starting Kubernetes Kubelet... Sep 18 03:17:41.093661 ip-10-0-129-90 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 03:17:41.093661 ip-10-0-129-90 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 18 03:17:41.093661 ip-10-0-129-90 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 03:17:41.093661 ip-10-0-129-90 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 18 03:17:41.093661 ip-10-0-129-90 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 03:17:41.094702 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.094638 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 18 03:17:41.101432 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101417 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:17:41.101432 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101432 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101436 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101439 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101442 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101445 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101449 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101451 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101455 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101460 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101463 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101465 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101468 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101470 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101473 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101475 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101478 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101480 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101483 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101485 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:17:41.101492 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101488 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101490 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101493 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101496 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101498 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101502 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101504 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101507 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101510 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101513 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101515 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101518 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101520 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101524 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101526 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101529 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101531 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101533 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101536 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101538 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:17:41.101941 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101541 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101544 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101546 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101548 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101551 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101553 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101556 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101558 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101561 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101563 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101565 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101568 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101570 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101573 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101576 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101578 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101581 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101584 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101586 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101589 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:17:41.102444 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101591 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101593 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101596 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101598 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101601 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101603 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101605 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101608 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101610 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101613 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101615 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101618 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101621 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101623 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101626 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101628 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101632 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101637 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101639 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101642 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:17:41.102923 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101644 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101647 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101650 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101652 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101654 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101658 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101662 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101666 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101670 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101673 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101675 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101678 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101682 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101685 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101688 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101691 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101694 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101696 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.101699 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:17:41.103438 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102393 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102400 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102403 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102406 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102410 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102414 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102417 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102420 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102422 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102425 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102427 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102430 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102433 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102435 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102438 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102441 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102443 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102446 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102449 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102452 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:17:41.103902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102454 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102457 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102460 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102462 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102465 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102467 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102470 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102472 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102475 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102477 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102480 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102482 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102484 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102487 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102489 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102492 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102494 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102497 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102500 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102503 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:17:41.104421 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102505 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102507 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102511 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102515 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102518 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102521 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102523 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102526 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102529 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102533 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102536 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102538 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102541 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102544 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102546 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102549 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102551 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102554 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102558 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:17:41.104902 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102562 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102564 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102567 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102571 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102574 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102576 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102579 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102581 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102584 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102586 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102588 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102591 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102594 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102596 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102599 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102601 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102603 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102606 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102609 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102611 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:17:41.105381 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102613 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102616 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102619 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102621 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102626 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102628 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102632 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102634 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102636 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102639 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102641 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102644 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102646 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102649 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102651 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102654 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102657 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102659 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102661 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.102664 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:17:41.105859 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103008 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103013 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103015 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103017 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103021 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103023 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103026 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103029 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103032 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103034 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103037 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103040 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103044 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103065 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103069 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103072 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103075 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103078 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103082 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:17:41.106371 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103084 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103087 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103090 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103092 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103095 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103098 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103100 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103103 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103105 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103108 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103110 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103112 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103115 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103118 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103120 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103123 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103125 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103128 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103137 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103139 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:17:41.106851 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103142 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103144 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103147 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103149 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103151 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103153 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103156 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103161 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103163 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103166 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103168 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103171 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103174 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103176 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103178 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103181 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103183 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103186 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103188 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103191 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:17:41.107340 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103193 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103196 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103198 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103201 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103203 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103206 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103208 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103210 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103213 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103215 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103217 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103221 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103223 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103226 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103228 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103231 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103233 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103236 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103238 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:17:41.107818 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103242 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103244 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103247 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103250 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103254 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103257 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103260 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103264 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103268 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103271 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103274 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103276 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103279 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103281 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103284 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103286 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103288 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103291 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103293 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:17:41.108323 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103296 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.103298 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103363 2580 flags.go:64] FLAG: --address="0.0.0.0" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103396 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103401 2580 flags.go:64] FLAG: --anonymous-auth="true" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103406 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103411 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103414 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103418 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103422 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103424 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103431 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103435 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103440 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103443 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103447 2580 flags.go:64] FLAG: --cgroup-root="" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103450 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103453 2580 flags.go:64] FLAG: --client-ca-file="" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103456 2580 flags.go:64] FLAG: --cloud-provider="external" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103459 2580 flags.go:64] FLAG: --cluster-dns="[]" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103466 2580 flags.go:64] FLAG: --cluster-domain="" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103469 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103472 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103475 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 18 03:17:41.108778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103478 2580 flags.go:64] FLAG: --container-log-max-files="5" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103482 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103485 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103487 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103491 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103493 2580 flags.go:64] FLAG: --contention-profiling="false" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103496 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103499 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103502 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103505 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103509 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103511 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103515 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103517 2580 flags.go:64] FLAG: --enable-load-reader="false" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103520 2580 flags.go:64] FLAG: --enable-server="true" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103523 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103527 2580 flags.go:64] FLAG: --event-burst="100" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103530 2580 flags.go:64] FLAG: --event-qps="50" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103533 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103535 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103538 2580 flags.go:64] FLAG: --eviction-hard="" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103542 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103545 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103548 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103551 2580 flags.go:64] FLAG: --eviction-soft="" Sep 18 03:17:41.109435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103554 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103557 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103559 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103562 2580 flags.go:64] FLAG: --experimental-mounter-path="" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103565 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103568 2580 flags.go:64] FLAG: --fail-swap-on="true" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103570 2580 flags.go:64] FLAG: --feature-gates="" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103574 2580 flags.go:64] FLAG: --file-check-frequency="20s" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103577 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103580 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103583 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103585 2580 flags.go:64] FLAG: --healthz-port="10248" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103588 2580 flags.go:64] FLAG: --help="false" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103591 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103594 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103597 2580 flags.go:64] FLAG: --http-check-frequency="20s" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103599 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103603 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103606 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103609 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103611 2580 flags.go:64] FLAG: --image-service-endpoint="" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103614 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103617 2580 flags.go:64] FLAG: --kube-api-burst="100" Sep 18 03:17:41.110034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103620 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103623 2580 flags.go:64] FLAG: --kube-api-qps="50" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103626 2580 flags.go:64] FLAG: --kube-reserved="" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103629 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103631 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103634 2580 flags.go:64] FLAG: --kubelet-cgroups="" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103637 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103651 2580 flags.go:64] FLAG: --lock-file="" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103654 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103657 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103661 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103666 2580 flags.go:64] FLAG: --log-json-split-stream="false" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103669 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103671 2580 flags.go:64] FLAG: --log-text-split-stream="false" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103674 2580 flags.go:64] FLAG: --logging-format="text" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103677 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103680 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103683 2580 flags.go:64] FLAG: --manifest-url="" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103685 2580 flags.go:64] FLAG: --manifest-url-header="" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103690 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103693 2580 flags.go:64] FLAG: --max-open-files="1000000" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103697 2580 flags.go:64] FLAG: --max-pods="110" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103699 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103702 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103705 2580 flags.go:64] FLAG: --memory-manager-policy="None" Sep 18 03:17:41.110716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103708 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103711 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103713 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103716 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103722 2580 flags.go:64] FLAG: --node-status-max-images="50" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103725 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103727 2580 flags.go:64] FLAG: --oom-score-adj="-999" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103730 2580 flags.go:64] FLAG: --pod-cidr="" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103734 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103739 2580 flags.go:64] FLAG: --pod-manifest-path="" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103742 2580 flags.go:64] FLAG: --pod-max-pids="-1" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103745 2580 flags.go:64] FLAG: --pods-per-core="0" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103748 2580 flags.go:64] FLAG: --port="10250" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103750 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103753 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04b029c41222a024f" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103756 2580 flags.go:64] FLAG: --qos-reserved="" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103759 2580 flags.go:64] FLAG: --read-only-port="10255" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103762 2580 flags.go:64] FLAG: --register-node="true" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103765 2580 flags.go:64] FLAG: --register-with-taints="" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103768 2580 flags.go:64] FLAG: --registry-burst="10" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103771 2580 flags.go:64] FLAG: --registry-qps="5" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103773 2580 flags.go:64] FLAG: --reserved-cpus="" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103776 2580 flags.go:64] FLAG: --reserved-memory="" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103779 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103782 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 18 03:17:41.111374 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103785 2580 flags.go:64] FLAG: --rotate-certificates="false" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103787 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103790 2580 flags.go:64] FLAG: --runonce="false" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103792 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103795 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103798 2580 flags.go:64] FLAG: --seccomp-default="false" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103801 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103803 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103806 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103809 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103812 2580 flags.go:64] FLAG: --storage-driver-password="root" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103814 2580 flags.go:64] FLAG: --storage-driver-secure="false" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103817 2580 flags.go:64] FLAG: --storage-driver-table="stats" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103819 2580 flags.go:64] FLAG: --storage-driver-user="root" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103822 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103825 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103828 2580 flags.go:64] FLAG: --system-cgroups="" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103831 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103836 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103839 2580 flags.go:64] FLAG: --tls-cert-file="" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103841 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103845 2580 flags.go:64] FLAG: --tls-min-version="" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103849 2580 flags.go:64] FLAG: --tls-private-key-file="" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103863 2580 flags.go:64] FLAG: --topology-manager-policy="none" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103865 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 18 03:17:41.112004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103869 2580 flags.go:64] FLAG: --topology-manager-scope="container" Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103872 2580 flags.go:64] FLAG: --v="2" Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103880 2580 flags.go:64] FLAG: --version="false" Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103884 2580 flags.go:64] FLAG: --vmodule="" Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103888 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.103891 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104005 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104009 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104012 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104015 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104018 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104021 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104023 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104026 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104028 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104031 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104033 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104036 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104038 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104041 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104043 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:17:41.112618 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104046 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104063 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104067 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104071 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104075 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104078 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104081 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104084 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104088 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104091 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104094 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104097 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104101 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104105 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104108 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104111 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104113 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104116 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104119 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:17:41.113185 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104122 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104125 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104128 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104130 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104133 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104136 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104139 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104142 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104145 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104147 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104150 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104152 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104155 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104157 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104160 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104163 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104165 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104169 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104172 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104174 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:17:41.113652 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104177 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104181 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104184 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104186 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104189 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104191 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104194 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104196 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104199 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104201 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104204 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104206 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104209 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104211 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104214 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104216 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104219 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104221 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104224 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104226 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:17:41.114173 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104229 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104231 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104234 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104236 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104239 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104242 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104244 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104246 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104250 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104255 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104258 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104261 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104264 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104269 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104271 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104274 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104276 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104279 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104281 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104284 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:17:41.114657 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104286 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104289 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104291 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104294 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.104304 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.105000 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.112416 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.112429 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.113516 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.113524 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113581 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113587 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113590 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113594 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113597 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:17:41.115227 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113600 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113602 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113605 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113607 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113610 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113613 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113616 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113621 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113625 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113628 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113631 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113634 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113637 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113639 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113642 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113645 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113647 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113650 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113652 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:17:41.115595 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113655 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113658 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113660 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113663 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113665 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113669 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113672 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113675 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113677 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113680 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113683 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113685 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113688 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113691 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113693 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113696 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113699 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113702 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113704 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113707 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:17:41.116071 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113709 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113712 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113715 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113717 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113720 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113722 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113726 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113729 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113732 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113735 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113737 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113740 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113742 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113746 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113749 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113752 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113754 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113758 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113761 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:17:41.116593 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113763 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113766 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113769 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113771 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113774 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113776 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113779 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113781 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113784 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113786 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113789 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113791 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113794 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113796 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113798 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113801 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113804 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113806 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113809 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113811 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:17:41.117062 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113813 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113816 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113819 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113821 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113824 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113826 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113829 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113831 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113834 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113837 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113839 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113842 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113845 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113847 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113850 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:17:41.117542 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113853 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.113857 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113965 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113969 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113972 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113975 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113978 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113981 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113985 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113987 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113990 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113992 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113995 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.113998 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114000 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 03:17:41.117899 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114003 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114005 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114008 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114010 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114013 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114015 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114017 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114020 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114022 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114025 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114027 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114030 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114033 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114036 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114038 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114040 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114043 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114046 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114067 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114070 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 03:17:41.118284 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114073 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114076 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114078 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114080 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114083 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114085 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114088 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114090 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114094 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114097 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114100 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114104 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114106 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114109 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114111 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114114 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114116 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114118 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114121 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 03:17:41.118765 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114123 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114126 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114128 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114131 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114133 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114137 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114141 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114144 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114147 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114150 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114152 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114155 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114158 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114161 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114163 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114165 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114168 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114171 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114173 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 03:17:41.119250 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114176 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114178 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114181 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114184 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114186 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114189 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114192 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114194 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114197 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114199 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114201 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114204 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114206 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114209 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114211 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114214 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114216 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114219 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114222 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114224 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 03:17:41.119708 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114227 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114229 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114232 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114234 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114236 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114239 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114241 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.114244 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.114249 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.114342 2580 server.go:962] "Client rotation is on, will bootstrap in background" Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.116876 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.117938 2580 server.go:1019] "Starting client certificate rotation" Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.118039 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 03:17:41.120566 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.118100 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 03:17:41.122534 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.122513 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 03:17:41.127584 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.127567 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 03:17:41.134884 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.134868 2580 log.go:25] "Validated CRI v1 runtime API" Sep 18 03:17:41.138335 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.138322 2580 log.go:25] "Validated CRI v1 image API" Sep 18 03:17:41.140130 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.140109 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 18 03:17:41.143618 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.143594 2580 fs.go:135] Filesystem UUIDs: map[6086c8e6-34ee-436a-a641-7ad5e8c2ba0d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c0a5303b-0d50-4e51-8cb6-28db1bfba7ad:/dev/nvme0n1p4] Sep 18 03:17:41.143681 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.143619 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 18 03:17:41.149202 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.149071 2580 manager.go:216] Machine: {Timestamp:2026-09-18 03:17:41.147615982 +0000 UTC m=+0.363057148 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103834 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b0f51d139a062a1e8318ac6637b90 SystemUUID:ec2b0f51-d139-a062-a1e8-318ac6637b90 BootID:409e560a-a004-4355-9e24-4ef5f52a645a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:cb:38:81:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:cb:38:81:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:f8:cd:e2:f9:fa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 18 03:17:41.149202 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.149194 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 18 03:17:41.149347 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.149285 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 18 03:17:41.151661 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.151629 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 03:17:41.152719 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.152699 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 18 03:17:41.152854 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.152722 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-90.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 18 03:17:41.153684 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.153672 2580 topology_manager.go:138] "Creating topology manager with none policy" Sep 18 03:17:41.153684 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.153684 2580 container_manager_linux.go:309] "Creating device plugin manager" Sep 18 03:17:41.153754 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.153691 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 03:17:41.154507 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.154498 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 03:17:41.154568 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.154560 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 18 03:17:41.159217 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.159206 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 18 03:17:41.159335 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.159326 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 18 03:17:41.159376 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.159357 2580 kubelet.go:486] "Attempting to sync node with API server" Sep 18 03:17:41.159376 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.159371 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 18 03:17:41.159456 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.159386 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 18 03:17:41.159456 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.159400 2580 kubelet.go:398] "Adding apiserver pod source" Sep 18 03:17:41.159456 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.159411 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 18 03:17:41.159736 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.159724 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 03:17:41.159770 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.159745 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 03:17:41.162774 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.162750 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 18 03:17:41.165736 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.165724 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 18 03:17:41.165791 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.165753 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 18 03:17:41.167044 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167032 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 18 03:17:41.167111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167061 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 18 03:17:41.167111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167068 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 18 03:17:41.167111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167077 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 18 03:17:41.167111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167089 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 18 03:17:41.167111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167095 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 18 03:17:41.167111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167108 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 18 03:17:41.167111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167113 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 18 03:17:41.167301 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167119 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 18 03:17:41.167301 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167125 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 18 03:17:41.167301 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167132 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 18 03:17:41.167301 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167137 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 18 03:17:41.167994 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167983 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 18 03:17:41.168033 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.167996 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 18 03:17:41.172521 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.172457 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-90.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 18 03:17:41.172562 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.172510 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-90.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 18 03:17:41.172562 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.172544 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 18 03:17:41.173493 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.173477 2580 server.go:1268] "Started kubelet" Sep 18 03:17:41.173639 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.173602 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 18 03:17:41.173691 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.173631 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 18 03:17:41.173752 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.173664 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 18 03:17:41.174132 ip-10-0-129-90 systemd[1]: Started Kubernetes Kubelet. Sep 18 03:17:41.174809 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.174794 2580 server.go:310] "Adding debug handlers to kubelet server" Sep 18 03:17:41.177246 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.177229 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 18 03:17:41.177596 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.177581 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 18 03:17:41.182441 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.181373 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-90.ec2.internal.18d64bff273db321 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-90.ec2.internal,UID:ip-10-0-129-90.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-90.ec2.internal,},FirstTimestamp:2026-09-18 03:17:41.173449505 +0000 UTC m=+0.388890672,LastTimestamp:2026-09-18 03:17:41.173449505 +0000 UTC m=+0.388890672,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-90.ec2.internal,}" Sep 18 03:17:41.184879 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.184863 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 18 03:17:41.185528 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.185512 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 18 03:17:41.185684 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.185666 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:41.186341 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.186327 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 18 03:17:41.186341 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.186341 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 18 03:17:41.186480 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.186328 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 18 03:17:41.186480 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.186477 2580 reconstruct.go:97] "Volume reconstruction finished" Sep 18 03:17:41.186536 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.186485 2580 reconciler.go:29] "Reconciler: start to sync state" Sep 18 03:17:41.187834 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.187813 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-72gvq" Sep 18 03:17:41.188132 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.188108 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 18 03:17:41.188207 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.188160 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-90.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 18 03:17:41.188207 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.188174 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 18 03:17:41.188207 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.188187 2580 factory.go:55] Registering systemd factory Sep 18 03:17:41.188207 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.188194 2580 factory.go:223] Registration of the systemd container factory successfully Sep 18 03:17:41.188409 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.188365 2580 factory.go:153] Registering CRI-O factory Sep 18 03:17:41.188409 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.188373 2580 factory.go:223] Registration of the crio container factory successfully Sep 18 03:17:41.188409 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.188390 2580 factory.go:103] Registering Raw factory Sep 18 03:17:41.188409 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.188401 2580 manager.go:1196] Started watching for new ooms in manager Sep 18 03:17:41.189025 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.189008 2580 manager.go:349] Starting recovery of all containers Sep 18 03:17:41.193143 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:41.193118 2580 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service/memory.swap.max": read /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service/memory.swap.max: no such device Sep 18 03:17:41.196563 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.196542 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-72gvq" Sep 18 03:17:41.199666 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.199651 2580 manager.go:354] Recovery completed Sep 18 03:17:41.203650 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.203628 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:17:41.205217 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.205202 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:17:41.205279 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.205227 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:17:41.205279 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.205237 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:17:41.205696 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.205679 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 18 03:17:41.205696 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.205694 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 18 03:17:41.205844 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.205710 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 18 03:17:41.205844 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.205733 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:17:41.206402 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.206386 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:17:41.206445 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.206415 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:17:41.206445 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.206425 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:17:41.207691 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.207678 2580 policy_none.go:49] "None policy: Start" Sep 18 03:17:41.207756 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.207696 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 18 03:17:41.207756 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.207709 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 18 03:17:41.209726 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.209713 2580 policy_none.go:47] "Start" Sep 18 03:17:41.241650 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.241636 2580 manager.go:337] "Starting Device Plugin manager" Sep 18 03:17:41.268380 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.241677 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 18 03:17:41.268380 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.241689 2580 server.go:87] "Starting device plugin registration server" Sep 18 03:17:41.268380 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.241893 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 18 03:17:41.268380 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.241904 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 18 03:17:41.268380 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.241993 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 18 03:17:41.268380 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.242079 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 18 03:17:41.268380 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.242088 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 18 03:17:41.268380 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.242491 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 18 03:17:41.268380 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.242532 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:41.291390 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.291371 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 18 03:17:41.292595 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.292581 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 18 03:17:41.292660 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.292623 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 18 03:17:41.292660 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.292642 2580 kubelet.go:2448] "Starting kubelet main sync loop" Sep 18 03:17:41.292775 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.292674 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 18 03:17:41.294903 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.294877 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:17:41.342661 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.342641 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:17:41.343394 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.343378 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:17:41.343466 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.343406 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:17:41.343466 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.343416 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:17:41.343466 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.343438 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.348877 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.348844 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.348877 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.348862 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-90.ec2.internal\": node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:41.359150 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.359129 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:41.393030 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.393008 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal"] Sep 18 03:17:41.393110 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.393097 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:17:41.394340 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.394327 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:17:41.394400 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.394354 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:17:41.394400 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.394369 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:17:41.396643 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.396631 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:17:41.396764 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.396749 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.396800 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.396777 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:17:41.397397 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.397383 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:17:41.397468 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.397408 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:17:41.397468 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.397423 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:17:41.397912 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.397898 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:17:41.397986 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.397926 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:17:41.397986 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.397939 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:17:41.400286 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.400270 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.400368 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.400300 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 03:17:41.401130 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.401115 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientMemory" Sep 18 03:17:41.401223 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.401144 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 03:17:41.401223 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.401158 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeHasSufficientPID" Sep 18 03:17:41.421645 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.421626 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-90.ec2.internal\" not found" node="ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.425792 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.425777 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-90.ec2.internal\" not found" node="ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.460100 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.460082 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:41.488207 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.488187 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a44ec86bb6ecf4f75a2e225ac81e21a1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal\" (UID: \"a44ec86bb6ecf4f75a2e225ac81e21a1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.488274 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.488211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a44ec86bb6ecf4f75a2e225ac81e21a1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal\" (UID: \"a44ec86bb6ecf4f75a2e225ac81e21a1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.488274 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.488246 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c5d8fca0c5700e570250443bdef4a41a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-90.ec2.internal\" (UID: \"c5d8fca0c5700e570250443bdef4a41a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.560284 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.560266 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:41.589244 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.589226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a44ec86bb6ecf4f75a2e225ac81e21a1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal\" (UID: \"a44ec86bb6ecf4f75a2e225ac81e21a1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.589304 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.589269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a44ec86bb6ecf4f75a2e225ac81e21a1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal\" (UID: \"a44ec86bb6ecf4f75a2e225ac81e21a1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.589304 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.589286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c5d8fca0c5700e570250443bdef4a41a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-90.ec2.internal\" (UID: \"c5d8fca0c5700e570250443bdef4a41a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.589405 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.589345 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c5d8fca0c5700e570250443bdef4a41a-config\") pod \"kube-apiserver-proxy-ip-10-0-129-90.ec2.internal\" (UID: \"c5d8fca0c5700e570250443bdef4a41a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.589405 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.589353 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a44ec86bb6ecf4f75a2e225ac81e21a1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal\" (UID: \"a44ec86bb6ecf4f75a2e225ac81e21a1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.589405 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.589363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a44ec86bb6ecf4f75a2e225ac81e21a1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal\" (UID: \"a44ec86bb6ecf4f75a2e225ac81e21a1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.660664 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.660609 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:41.726114 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.726094 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.729198 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:41.729180 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" Sep 18 03:17:41.761248 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.761224 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:41.861744 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.861710 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:41.962280 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:41.962215 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:42.062791 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:42.062767 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:42.117329 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.117303 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 18 03:17:42.117695 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.117429 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 18 03:17:42.163892 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:42.163770 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-90.ec2.internal\" not found" Sep 18 03:17:42.204129 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.204102 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 03:12:41 +0000 UTC" deadline="2028-03-23 22:28:55.531038955 +0000 UTC" Sep 18 03:17:42.204129 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.204127 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13267h11m13.326914732s" Sep 18 03:17:42.209287 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.209264 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:17:42.286132 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.286084 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal" Sep 18 03:17:42.298569 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.298462 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 03:17:42.302143 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.302122 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" Sep 18 03:17:42.316159 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.316135 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 03:17:42.323838 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:42.323809 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda44ec86bb6ecf4f75a2e225ac81e21a1.slice/crio-a6a4eb8b4aed1cc035a22f467b62c17b072b3af0c21978a66750b315490c7ee7 WatchSource:0}: Error finding container a6a4eb8b4aed1cc035a22f467b62c17b072b3af0c21978a66750b315490c7ee7: Status 404 returned error can't find the container with id a6a4eb8b4aed1cc035a22f467b62c17b072b3af0c21978a66750b315490c7ee7 Sep 18 03:17:42.324328 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:42.324305 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d8fca0c5700e570250443bdef4a41a.slice/crio-c8cebba38cee11a4cc455594150600c5274aed7b2e37827ef05121878871a4e3 WatchSource:0}: Error finding container c8cebba38cee11a4cc455594150600c5274aed7b2e37827ef05121878871a4e3: Status 404 returned error can't find the container with id c8cebba38cee11a4cc455594150600c5274aed7b2e37827ef05121878871a4e3 Sep 18 03:17:42.329782 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.329770 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 03:17:42.510742 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.510722 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:17:42.678777 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:42.678750 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:17:43.160518 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.160490 2580 apiserver.go:52] "Watching apiserver" Sep 18 03:17:43.171191 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.171168 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 18 03:17:43.172373 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.172348 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-fzpz8","kube-system/konnectivity-agent-b8sbn","kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal","openshift-multus/network-metrics-daemon-ssqd2","openshift-network-diagnostics/network-check-target-hwg2m","openshift-network-operator/iptables-alerter-t2nmd","openshift-ovn-kubernetes/ovnkube-node-s8m7g","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh","openshift-cluster-node-tuning-operator/tuned-pbf5f","openshift-dns/node-resolver-d29f9","openshift-image-registry/node-ca-fdnrl","openshift-multus/multus-additional-cni-plugins-wvlk7","openshift-multus/multus-fldjl"] Sep 18 03:17:43.175103 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.175084 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:43.175214 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.175169 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:17:43.177449 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.177428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.180210 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.180187 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.182701 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.182220 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.183196 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.183005 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 18 03:17:43.183196 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.183041 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 18 03:17:43.183841 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.183822 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 18 03:17:43.183939 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.183873 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wxv2c\"" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185005 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185224 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j65mg\"" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185303 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185432 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185460 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185569 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185227 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185684 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.185726 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:17:43.185820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185763 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 18 03:17:43.186421 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.185856 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 18 03:17:43.186421 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.186388 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 18 03:17:43.186667 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.186552 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 18 03:17:43.186915 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.186892 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 18 03:17:43.187114 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.187098 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5jwmc\"" Sep 18 03:17:43.187900 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.187879 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.188007 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.187947 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:17:43.190521 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.190500 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.193772 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.193586 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 18 03:17:43.193772 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.193619 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-chjjc\"" Sep 18 03:17:43.194800 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.194776 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.195882 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.195661 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 18 03:17:43.196516 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.196404 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 18 03:17:43.197248 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-var-lib-openvswitch\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197330 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197267 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-etc-openvswitch\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197330 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197294 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-cni-netd\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197330 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197319 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq4qq\" (UniqueName: \"kubernetes.io/projected/9c20f405-c42c-4c5b-b789-465af48ce967-kube-api-access-dq4qq\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197485 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197347 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-run-netns\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.197485 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-run-netns\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197485 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197398 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c20f405-c42c-4c5b-b789-465af48ce967-ovn-node-metrics-cert\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197485 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnhpl\" (UniqueName: \"kubernetes.io/projected/a1091cd1-b35a-49b0-b97c-a15ee4efbebe-kube-api-access-lnhpl\") pod \"iptables-alerter-t2nmd\" (UID: \"a1091cd1-b35a-49b0-b97c-a15ee4efbebe\") " pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.197485 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197445 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-run-k8s-cni-cncf-io\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.197485 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-var-lib-cni-bin\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.197485 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197449 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197499 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-var-lib-cni-multus\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-systemd-units\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-cni-bin\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197575 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c20f405-c42c-4c5b-b789-465af48ce967-ovnkube-script-lib\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197612 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-multus-socket-dir-parent\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-var-lib-kubelet\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197662 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-slash\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-run-openvswitch\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197727 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c20f405-c42c-4c5b-b789-465af48ce967-ovnkube-config\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197773 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c20f405-c42c-4c5b-b789-465af48ce967-env-overrides\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48c50c68-f633-4e98-8823-587153372c93-multus-daemon-config\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197823 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-run-systemd\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.197846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-node-log\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197873 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a1091cd1-b35a-49b0-b97c-a15ee4efbebe-iptables-alerter-script\") pod \"iptables-alerter-t2nmd\" (UID: \"a1091cd1-b35a-49b0-b97c-a15ee4efbebe\") " pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1091cd1-b35a-49b0-b97c-a15ee4efbebe-host-slash\") pod \"iptables-alerter-t2nmd\" (UID: \"a1091cd1-b35a-49b0-b97c-a15ee4efbebe\") " pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197920 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/994a76dc-2129-4b42-8bb2-a51be4c091d7-dbus\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197946 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhcd2\" (UniqueName: \"kubernetes.io/projected/48c50c68-f633-4e98-8823-587153372c93-kube-api-access-hhcd2\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.197996 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198014 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198021 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-multus-conf-dir\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-etc-kubernetes\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrzrh\" (UniqueName: \"kubernetes.io/projected/ffd6a923-32d3-4212-8231-20b173a6a6a7-kube-api-access-lrzrh\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/994a76dc-2129-4b42-8bb2-a51be4c091d7-kubelet-config\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198194 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-multus-cni-dir\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-cnibin\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-kubelet\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198255 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-99vss\"" Sep 18 03:17:43.198320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198262 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 18 03:17:43.199185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-run-ovn\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.199185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198296 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-log-socket\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.199185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-system-cni-dir\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.199185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198341 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48c50c68-f633-4e98-8823-587153372c93-cni-binary-copy\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.199185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198366 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-hostroot\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.199185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198389 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-run-multus-certs\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.199185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.198568 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 03:17:43.199991 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.199971 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.200411 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.200390 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 18 03:17:43.200619 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.200599 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 18 03:17:43.200850 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.200830 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bzlgv\"" Sep 18 03:17:43.202873 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.202854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.203375 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.203347 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 18 03:17:43.203375 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.203371 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 18 03:17:43.203559 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.203442 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q5b2p\"" Sep 18 03:17:43.203559 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.203508 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 18 03:17:43.205023 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.205001 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:17:43.205778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.205758 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 18 03:17:43.206263 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.206246 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 18 03:17:43.206521 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.206274 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qkjsl\"" Sep 18 03:17:43.208111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.207943 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sk242\"" Sep 18 03:17:43.208111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.207948 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 18 03:17:43.208111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.208061 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 18 03:17:43.227763 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.227698 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fmfd4" Sep 18 03:17:43.237654 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.237634 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fmfd4" Sep 18 03:17:43.271190 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.271169 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 03:17:43.287579 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.287559 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 18 03:17:43.304310 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhcd2\" (UniqueName: \"kubernetes.io/projected/48c50c68-f633-4e98-8823-587153372c93-kube-api-access-hhcd2\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.304417 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304326 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-etc-selinux\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.304417 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304347 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7af14471-f55a-469b-a673-9f9903c399e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.304417 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjr9h\" (UniqueName: \"kubernetes.io/projected/7af14471-f55a-469b-a673-9f9903c399e3-kube-api-access-kjr9h\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.304417 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-multus-conf-dir\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.304645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-etc-kubernetes\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.304645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304449 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:43.304645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.304645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-sysctl-d\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.304645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-lib-modules\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.304645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-var-lib-kubelet\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.304645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-multus-cni-dir\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.304645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304614 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-kubelet\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.304645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-run-ovn\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304657 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-registration-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304684 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-sysconfig\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-systemd\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304720 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-tuned\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-system-cni-dir\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304768 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-etc-openvswitch\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq4qq\" (UniqueName: \"kubernetes.io/projected/9c20f405-c42c-4c5b-b789-465af48ce967-kube-api-access-dq4qq\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304819 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-sysctl-conf\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304839 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-run\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-run-k8s-cni-cncf-io\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-var-lib-cni-bin\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304913 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-systemd-units\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-socket-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dddf8f39-e67b-48a2-bac4-2399e3718e57-tmp\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-var-lib-kubelet\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.304993 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-slash\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.305082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c20f405-c42c-4c5b-b789-465af48ce967-env-overrides\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-sys\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgbr6\" (UniqueName: \"kubernetes.io/projected/dddf8f39-e67b-48a2-bac4-2399e3718e57-kube-api-access-vgbr6\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305092 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbbg7\" (UniqueName: \"kubernetes.io/projected/5b477d64-4f58-4f2b-aca6-f898589820cb-kube-api-access-fbbg7\") pod \"node-ca-fdnrl\" (UID: \"5b477d64-4f58-4f2b-aca6-f898589820cb\") " pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-run-systemd\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-node-log\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1091cd1-b35a-49b0-b97c-a15ee4efbebe-host-slash\") pod \"iptables-alerter-t2nmd\" (UID: \"a1091cd1-b35a-49b0-b97c-a15ee4efbebe\") " pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/994a76dc-2129-4b42-8bb2-a51be4c091d7-dbus\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtlpc\" (UniqueName: \"kubernetes.io/projected/578ed779-0fd5-4517-91b4-58b82bcc8a97-kube-api-access-qtlpc\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305262 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9wj8\" (UniqueName: \"kubernetes.io/projected/b2814e15-58e8-4144-becf-e6af9314b295-kube-api-access-w9wj8\") pod \"node-resolver-d29f9\" (UID: \"b2814e15-58e8-4144-becf-e6af9314b295\") " pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305281 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7af14471-f55a-469b-a673-9f9903c399e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305300 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3b737b9-7cab-470a-8c89-d20c07544375-agent-certs\") pod \"konnectivity-agent-b8sbn\" (UID: \"b3b737b9-7cab-470a-8c89-d20c07544375\") " pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305319 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305339 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-modprobe-d\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305377 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrzrh\" (UniqueName: \"kubernetes.io/projected/ffd6a923-32d3-4212-8231-20b173a6a6a7-kube-api-access-lrzrh\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:43.305813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/994a76dc-2129-4b42-8bb2-a51be4c091d7-kubelet-config\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305471 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7af14471-f55a-469b-a673-9f9903c399e3-system-cni-dir\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7af14471-f55a-469b-a673-9f9903c399e3-cnibin\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305530 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-host\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-cnibin\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-log-socket\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305585 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b477d64-4f58-4f2b-aca6-f898589820cb-host\") pod \"node-ca-fdnrl\" (UID: \"5b477d64-4f58-4f2b-aca6-f898589820cb\") " pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305620 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48c50c68-f633-4e98-8823-587153372c93-cni-binary-copy\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-hostroot\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305680 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-run-multus-certs\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-var-lib-openvswitch\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-cni-netd\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2814e15-58e8-4144-becf-e6af9314b295-hosts-file\") pod \"node-resolver-d29f9\" (UID: \"b2814e15-58e8-4144-becf-e6af9314b295\") " pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-kubernetes\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b477d64-4f58-4f2b-aca6-f898589820cb-serviceca\") pod \"node-ca-fdnrl\" (UID: \"5b477d64-4f58-4f2b-aca6-f898589820cb\") " pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-run-netns\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.306636 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305832 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-run-netns\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305853 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c20f405-c42c-4c5b-b789-465af48ce967-ovn-node-metrics-cert\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnhpl\" (UniqueName: \"kubernetes.io/projected/a1091cd1-b35a-49b0-b97c-a15ee4efbebe-kube-api-access-lnhpl\") pod \"iptables-alerter-t2nmd\" (UID: \"a1091cd1-b35a-49b0-b97c-a15ee4efbebe\") " pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305891 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-sys-fs\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-var-lib-cni-multus\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305936 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-cni-bin\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c20f405-c42c-4c5b-b789-465af48ce967-ovnkube-script-lib\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2814e15-58e8-4144-becf-e6af9314b295-tmp-dir\") pod \"node-resolver-d29f9\" (UID: \"b2814e15-58e8-4144-becf-e6af9314b295\") " pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.305993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7af14471-f55a-469b-a673-9f9903c399e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306033 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-multus-socket-dir-parent\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-run-openvswitch\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c20f405-c42c-4c5b-b789-465af48ce967-ovnkube-config\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7af14471-f55a-469b-a673-9f9903c399e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306153 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3b737b9-7cab-470a-8c89-d20c07544375-konnectivity-ca\") pod \"konnectivity-agent-b8sbn\" (UID: \"b3b737b9-7cab-470a-8c89-d20c07544375\") " pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306172 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48c50c68-f633-4e98-8823-587153372c93-multus-daemon-config\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.307221 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a1091cd1-b35a-49b0-b97c-a15ee4efbebe-iptables-alerter-script\") pod \"iptables-alerter-t2nmd\" (UID: \"a1091cd1-b35a-49b0-b97c-a15ee4efbebe\") " pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306219 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-device-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306339 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.306429 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.306461 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-etc-openvswitch\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.306492 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret podName:994a76dc-2129-4b42-8bb2-a51be4c091d7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:43.806474956 +0000 UTC m=+3.021916124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret") pod "global-pull-secret-syncer-fzpz8" (UID: "994a76dc-2129-4b42-8bb2-a51be4c091d7") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-etc-kubernetes\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307404 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-multus-socket-dir-parent\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307417 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-multus-conf-dir\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307444 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1091cd1-b35a-49b0-b97c-a15ee4efbebe-host-slash\") pod \"iptables-alerter-t2nmd\" (UID: \"a1091cd1-b35a-49b0-b97c-a15ee4efbebe\") " pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.307502 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307551 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/994a76dc-2129-4b42-8bb2-a51be4c091d7-dbus\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307591 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/994a76dc-2129-4b42-8bb2-a51be4c091d7-kubelet-config\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.307619 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs podName:ffd6a923-32d3-4212-8231-20b173a6a6a7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:43.807600898 +0000 UTC m=+3.023042068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs") pod "network-metrics-daemon-ssqd2" (UID: "ffd6a923-32d3-4212-8231-20b173a6a6a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307618 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307727 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-run-k8s-cni-cncf-io\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307733 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-hostroot\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.307955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307746 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-run-multus-certs\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307825 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-system-cni-dir\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307804 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-cni-bin\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.307855 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-run-ovn\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308093 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c20f405-c42c-4c5b-b789-465af48ce967-env-overrides\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308028 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-cnibin\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308185 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-log-socket\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308192 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-multus-cni-dir\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308232 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-kubelet\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308263 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-slash\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308286 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-systemd-units\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308331 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-var-lib-kubelet\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308336 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-var-lib-cni-bin\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a1091cd1-b35a-49b0-b97c-a15ee4efbebe-iptables-alerter-script\") pod \"iptables-alerter-t2nmd\" (UID: \"a1091cd1-b35a-49b0-b97c-a15ee4efbebe\") " pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308384 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-run-openvswitch\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308453 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-node-log\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308550 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-run-systemd\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308599 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-var-lib-cni-multus\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.308917 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308626 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-run-netns\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.309864 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308654 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-var-lib-openvswitch\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.309864 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308688 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c20f405-c42c-4c5b-b789-465af48ce967-ovnkube-config\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.309864 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308692 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c20f405-c42c-4c5b-b789-465af48ce967-host-cni-netd\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.309864 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308853 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48c50c68-f633-4e98-8823-587153372c93-multus-daemon-config\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.309864 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308950 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48c50c68-f633-4e98-8823-587153372c93-host-run-netns\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.309864 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.308943 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c20f405-c42c-4c5b-b789-465af48ce967-ovnkube-script-lib\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.309864 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.309105 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 18 03:17:43.309864 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.309670 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48c50c68-f633-4e98-8823-587153372c93-cni-binary-copy\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.311001 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.310954 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal" event={"ID":"c5d8fca0c5700e570250443bdef4a41a","Type":"ContainerStarted","Data":"c8cebba38cee11a4cc455594150600c5274aed7b2e37827ef05121878871a4e3"} Sep 18 03:17:43.312331 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.312304 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" event={"ID":"a44ec86bb6ecf4f75a2e225ac81e21a1","Type":"ContainerStarted","Data":"a6a4eb8b4aed1cc035a22f467b62c17b072b3af0c21978a66750b315490c7ee7"} Sep 18 03:17:43.313645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.313625 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c20f405-c42c-4c5b-b789-465af48ce967-ovn-node-metrics-cert\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.317291 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.317265 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:17:43.317376 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.317294 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:17:43.317576 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.317522 2580 projected.go:196] Error preparing data for projected volume kube-api-access-dk4tz for pod openshift-network-diagnostics/network-check-target-hwg2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:43.317746 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.317615 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz podName:f274a119-a4cd-4900-9ebc-4c3d7c023542 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:43.817596557 +0000 UTC m=+3.033037729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dk4tz" (UniqueName: "kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz") pod "network-check-target-hwg2m" (UID: "f274a119-a4cd-4900-9ebc-4c3d7c023542") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:43.319318 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.319273 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhcd2\" (UniqueName: \"kubernetes.io/projected/48c50c68-f633-4e98-8823-587153372c93-kube-api-access-hhcd2\") pod \"multus-fldjl\" (UID: \"48c50c68-f633-4e98-8823-587153372c93\") " pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.319550 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.319525 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq4qq\" (UniqueName: \"kubernetes.io/projected/9c20f405-c42c-4c5b-b789-465af48ce967-kube-api-access-dq4qq\") pod \"ovnkube-node-s8m7g\" (UID: \"9c20f405-c42c-4c5b-b789-465af48ce967\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.319789 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.319762 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrzrh\" (UniqueName: \"kubernetes.io/projected/ffd6a923-32d3-4212-8231-20b173a6a6a7-kube-api-access-lrzrh\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:43.320703 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.320672 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnhpl\" (UniqueName: \"kubernetes.io/projected/a1091cd1-b35a-49b0-b97c-a15ee4efbebe-kube-api-access-lnhpl\") pod \"iptables-alerter-t2nmd\" (UID: \"a1091cd1-b35a-49b0-b97c-a15ee4efbebe\") " pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.407208 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407179 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2814e15-58e8-4144-becf-e6af9314b295-hosts-file\") pod \"node-resolver-d29f9\" (UID: \"b2814e15-58e8-4144-becf-e6af9314b295\") " pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407215 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-kubernetes\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b477d64-4f58-4f2b-aca6-f898589820cb-serviceca\") pod \"node-ca-fdnrl\" (UID: \"5b477d64-4f58-4f2b-aca6-f898589820cb\") " pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-sys-fs\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2814e15-58e8-4144-becf-e6af9314b295-tmp-dir\") pod \"node-resolver-d29f9\" (UID: \"b2814e15-58e8-4144-becf-e6af9314b295\") " pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7af14471-f55a-469b-a673-9f9903c399e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407317 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2814e15-58e8-4144-becf-e6af9314b295-hosts-file\") pod \"node-resolver-d29f9\" (UID: \"b2814e15-58e8-4144-becf-e6af9314b295\") " pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7af14471-f55a-469b-a673-9f9903c399e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407336 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-kubernetes\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407341 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-sys-fs\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.407356 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3b737b9-7cab-470a-8c89-d20c07544375-konnectivity-ca\") pod \"konnectivity-agent-b8sbn\" (UID: \"b3b737b9-7cab-470a-8c89-d20c07544375\") " pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-device-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-etc-selinux\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7af14471-f55a-469b-a673-9f9903c399e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjr9h\" (UniqueName: \"kubernetes.io/projected/7af14471-f55a-469b-a673-9f9903c399e3-kube-api-access-kjr9h\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-sysctl-d\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-lib-modules\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407568 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-var-lib-kubelet\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407586 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-registration-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-sysconfig\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407625 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2814e15-58e8-4144-becf-e6af9314b295-tmp-dir\") pod \"node-resolver-d29f9\" (UID: \"b2814e15-58e8-4144-becf-e6af9314b295\") " pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407638 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-systemd\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-tuned\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-sysctl-conf\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407725 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-device-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-run\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.407869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407771 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-sysctl-d\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407771 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b477d64-4f58-4f2b-aca6-f898589820cb-serviceca\") pod \"node-ca-fdnrl\" (UID: \"5b477d64-4f58-4f2b-aca6-f898589820cb\") " pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-socket-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407816 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-registration-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407823 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dddf8f39-e67b-48a2-bac4-2399e3718e57-tmp\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407829 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-var-lib-kubelet\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407849 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-sysconfig\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-sysctl-conf\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407932 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-etc-selinux\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407850 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-sys\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407963 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-systemd\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407975 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7af14471-f55a-469b-a673-9f9903c399e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgbr6\" (UniqueName: \"kubernetes.io/projected/dddf8f39-e67b-48a2-bac4-2399e3718e57-kube-api-access-vgbr6\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407990 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-run\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408019 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbbg7\" (UniqueName: \"kubernetes.io/projected/5b477d64-4f58-4f2b-aca6-f898589820cb-kube-api-access-fbbg7\") pod \"node-ca-fdnrl\" (UID: \"5b477d64-4f58-4f2b-aca6-f898589820cb\") " pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408035 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-lib-modules\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408043 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3b737b9-7cab-470a-8c89-d20c07544375-konnectivity-ca\") pod \"konnectivity-agent-b8sbn\" (UID: \"b3b737b9-7cab-470a-8c89-d20c07544375\") " pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:17:43.408735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtlpc\" (UniqueName: \"kubernetes.io/projected/578ed779-0fd5-4517-91b4-58b82bcc8a97-kube-api-access-qtlpc\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408078 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9wj8\" (UniqueName: \"kubernetes.io/projected/b2814e15-58e8-4144-becf-e6af9314b295-kube-api-access-w9wj8\") pod \"node-resolver-d29f9\" (UID: \"b2814e15-58e8-4144-becf-e6af9314b295\") " pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7af14471-f55a-469b-a673-9f9903c399e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408142 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-sys\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3b737b9-7cab-470a-8c89-d20c07544375-agent-certs\") pod \"konnectivity-agent-b8sbn\" (UID: \"b3b737b9-7cab-470a-8c89-d20c07544375\") " pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408162 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-socket-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.407943 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7af14471-f55a-469b-a673-9f9903c399e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-modprobe-d\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408254 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/578ed779-0fd5-4517-91b4-58b82bcc8a97-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7af14471-f55a-469b-a673-9f9903c399e3-system-cni-dir\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7af14471-f55a-469b-a673-9f9903c399e3-cnibin\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408309 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7af14471-f55a-469b-a673-9f9903c399e3-system-cni-dir\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408348 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7af14471-f55a-469b-a673-9f9903c399e3-cnibin\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408452 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7af14471-f55a-469b-a673-9f9903c399e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408479 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-modprobe-d\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-host\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408483 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7af14471-f55a-469b-a673-9f9903c399e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.409810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b477d64-4f58-4f2b-aca6-f898589820cb-host\") pod \"node-ca-fdnrl\" (UID: \"5b477d64-4f58-4f2b-aca6-f898589820cb\") " pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.410709 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408564 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dddf8f39-e67b-48a2-bac4-2399e3718e57-host\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.410709 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.408604 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b477d64-4f58-4f2b-aca6-f898589820cb-host\") pod \"node-ca-fdnrl\" (UID: \"5b477d64-4f58-4f2b-aca6-f898589820cb\") " pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.410709 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.410677 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dddf8f39-e67b-48a2-bac4-2399e3718e57-tmp\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.410892 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.410754 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dddf8f39-e67b-48a2-bac4-2399e3718e57-etc-tuned\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.411181 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.411161 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3b737b9-7cab-470a-8c89-d20c07544375-agent-certs\") pod \"konnectivity-agent-b8sbn\" (UID: \"b3b737b9-7cab-470a-8c89-d20c07544375\") " pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:17:43.419940 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.419913 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtlpc\" (UniqueName: \"kubernetes.io/projected/578ed779-0fd5-4517-91b4-58b82bcc8a97-kube-api-access-qtlpc\") pod \"aws-ebs-csi-driver-node-tjvjh\" (UID: \"578ed779-0fd5-4517-91b4-58b82bcc8a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.420266 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.420237 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjr9h\" (UniqueName: \"kubernetes.io/projected/7af14471-f55a-469b-a673-9f9903c399e3-kube-api-access-kjr9h\") pod \"multus-additional-cni-plugins-wvlk7\" (UID: \"7af14471-f55a-469b-a673-9f9903c399e3\") " pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.420353 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.420321 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbbg7\" (UniqueName: \"kubernetes.io/projected/5b477d64-4f58-4f2b-aca6-f898589820cb-kube-api-access-fbbg7\") pod \"node-ca-fdnrl\" (UID: \"5b477d64-4f58-4f2b-aca6-f898589820cb\") " pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.420740 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.420720 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9wj8\" (UniqueName: \"kubernetes.io/projected/b2814e15-58e8-4144-becf-e6af9314b295-kube-api-access-w9wj8\") pod \"node-resolver-d29f9\" (UID: \"b2814e15-58e8-4144-becf-e6af9314b295\") " pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.421071 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.421037 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgbr6\" (UniqueName: \"kubernetes.io/projected/dddf8f39-e67b-48a2-bac4-2399e3718e57-kube-api-access-vgbr6\") pod \"tuned-pbf5f\" (UID: \"dddf8f39-e67b-48a2-bac4-2399e3718e57\") " pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.490241 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.490215 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fldjl" Sep 18 03:17:43.502561 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.502544 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:17:43.513673 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.513654 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t2nmd" Sep 18 03:17:43.519502 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.519484 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" Sep 18 03:17:43.528425 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.528405 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" Sep 18 03:17:43.536328 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.536310 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d29f9" Sep 18 03:17:43.546201 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.546179 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fdnrl" Sep 18 03:17:43.552950 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.552932 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" Sep 18 03:17:43.559662 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.559644 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:17:43.812276 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.812241 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:43.812435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.812309 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:43.812435 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.812420 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:43.812435 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.812427 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:43.812596 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.812494 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret podName:994a76dc-2129-4b42-8bb2-a51be4c091d7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:44.812474547 +0000 UTC m=+4.027915778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret") pod "global-pull-secret-syncer-fzpz8" (UID: "994a76dc-2129-4b42-8bb2-a51be4c091d7") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:43.812596 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.812513 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs podName:ffd6a923-32d3-4212-8231-20b173a6a6a7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:44.812502666 +0000 UTC m=+4.027943832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs") pod "network-metrics-daemon-ssqd2" (UID: "ffd6a923-32d3-4212-8231-20b173a6a6a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:43.912599 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:43.912563 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:43.912741 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.912696 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:17:43.912741 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.912714 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:17:43.912741 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.912724 2580 projected.go:196] Error preparing data for projected volume kube-api-access-dk4tz for pod openshift-network-diagnostics/network-check-target-hwg2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:43.912897 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:43.912779 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz podName:f274a119-a4cd-4900-9ebc-4c3d7c023542 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:44.912760029 +0000 UTC m=+4.128201205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dk4tz" (UniqueName: "kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz") pod "network-check-target-hwg2m" (UID: "f274a119-a4cd-4900-9ebc-4c3d7c023542") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:44.241414 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:44.241334 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 03:12:43 +0000 UTC" deadline="2028-05-10 13:04:57.62643165 +0000 UTC" Sep 18 03:17:44.241414 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:44.241363 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14409h47m13.385071365s" Sep 18 03:17:44.293916 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:44.293885 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:44.294105 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:44.294020 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:17:44.435701 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:44.435114 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578ed779_0fd5_4517_91b4_58b82bcc8a97.slice/crio-be5562444d6f1862ea1c4c7bf269369f0148d84ad7cef6658b300cbd4cef35ca WatchSource:0}: Error finding container be5562444d6f1862ea1c4c7bf269369f0148d84ad7cef6658b300cbd4cef35ca: Status 404 returned error can't find the container with id be5562444d6f1862ea1c4c7bf269369f0148d84ad7cef6658b300cbd4cef35ca Sep 18 03:17:44.437790 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:44.437724 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c20f405_c42c_4c5b_b789_465af48ce967.slice/crio-8fffe558774fb7ed81fc6a5886ed39c1416e7144a8a86d9ebeadc40ad71d1d36 WatchSource:0}: Error finding container 8fffe558774fb7ed81fc6a5886ed39c1416e7144a8a86d9ebeadc40ad71d1d36: Status 404 returned error can't find the container with id 8fffe558774fb7ed81fc6a5886ed39c1416e7144a8a86d9ebeadc40ad71d1d36 Sep 18 03:17:44.442254 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:44.442226 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2814e15_58e8_4144_becf_e6af9314b295.slice/crio-f46e35bb4466090792d7b191dbe31b7f1d29ad1dd23398469b1a155ea7fb7760 WatchSource:0}: Error finding container f46e35bb4466090792d7b191dbe31b7f1d29ad1dd23398469b1a155ea7fb7760: Status 404 returned error can't find the container with id f46e35bb4466090792d7b191dbe31b7f1d29ad1dd23398469b1a155ea7fb7760 Sep 18 03:17:44.443863 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:44.443827 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddddf8f39_e67b_48a2_bac4_2399e3718e57.slice/crio-fd76ffb8c9b817efd333796fec107f561767252f31e64c3a9fc179a58ff48bc4 WatchSource:0}: Error finding container fd76ffb8c9b817efd333796fec107f561767252f31e64c3a9fc179a58ff48bc4: Status 404 returned error can't find the container with id fd76ffb8c9b817efd333796fec107f561767252f31e64c3a9fc179a58ff48bc4 Sep 18 03:17:44.445736 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:44.445710 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c50c68_f633_4e98_8823_587153372c93.slice/crio-7f02deff1fc730babe5e3cd4060d78e80eb752ae7a45d32a116e4e9d75a0b6c4 WatchSource:0}: Error finding container 7f02deff1fc730babe5e3cd4060d78e80eb752ae7a45d32a116e4e9d75a0b6c4: Status 404 returned error can't find the container with id 7f02deff1fc730babe5e3cd4060d78e80eb752ae7a45d32a116e4e9d75a0b6c4 Sep 18 03:17:44.446878 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:44.446854 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b477d64_4f58_4f2b_aca6_f898589820cb.slice/crio-18cf262278a4037e4bf95ed06b37cdf4ff447afe0a5e9fa0702411fd46671799 WatchSource:0}: Error finding container 18cf262278a4037e4bf95ed06b37cdf4ff447afe0a5e9fa0702411fd46671799: Status 404 returned error can't find the container with id 18cf262278a4037e4bf95ed06b37cdf4ff447afe0a5e9fa0702411fd46671799 Sep 18 03:17:44.447599 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:44.447534 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1091cd1_b35a_49b0_b97c_a15ee4efbebe.slice/crio-c0e589457c104c4554960e13f30106a3ee08a535471c63dfbc3892abf9fce7fa WatchSource:0}: Error finding container c0e589457c104c4554960e13f30106a3ee08a535471c63dfbc3892abf9fce7fa: Status 404 returned error can't find the container with id c0e589457c104c4554960e13f30106a3ee08a535471c63dfbc3892abf9fce7fa Sep 18 03:17:44.449468 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:17:44.449326 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b737b9_7cab_470a_8c89_d20c07544375.slice/crio-cb87834925d3bcecee31e06c9284a1cb0626c99a1b0e72e3748bacc6b4cc1e34 WatchSource:0}: Error finding container cb87834925d3bcecee31e06c9284a1cb0626c99a1b0e72e3748bacc6b4cc1e34: Status 404 returned error can't find the container with id cb87834925d3bcecee31e06c9284a1cb0626c99a1b0e72e3748bacc6b4cc1e34 Sep 18 03:17:44.818323 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:44.818147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:44.818418 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:44.818371 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:44.818418 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:44.818294 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:44.818487 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:44.818458 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret podName:994a76dc-2129-4b42-8bb2-a51be4c091d7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:46.818438412 +0000 UTC m=+6.033879592 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret") pod "global-pull-secret-syncer-fzpz8" (UID: "994a76dc-2129-4b42-8bb2-a51be4c091d7") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:44.818487 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:44.818467 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:44.818562 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:44.818507 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs podName:ffd6a923-32d3-4212-8231-20b173a6a6a7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:46.81849392 +0000 UTC m=+6.033935080 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs") pod "network-metrics-daemon-ssqd2" (UID: "ffd6a923-32d3-4212-8231-20b173a6a6a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:44.919249 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:44.919222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:44.919406 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:44.919387 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:17:44.919478 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:44.919411 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:17:44.919478 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:44.919422 2580 projected.go:196] Error preparing data for projected volume kube-api-access-dk4tz for pod openshift-network-diagnostics/network-check-target-hwg2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:44.919573 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:44.919480 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz podName:f274a119-a4cd-4900-9ebc-4c3d7c023542 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:46.919463749 +0000 UTC m=+6.134904912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dk4tz" (UniqueName: "kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz") pod "network-check-target-hwg2m" (UID: "f274a119-a4cd-4900-9ebc-4c3d7c023542") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:45.242240 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.242175 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 03:12:43 +0000 UTC" deadline="2028-05-21 16:39:53.057334248 +0000 UTC" Sep 18 03:17:45.242240 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.242237 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14677h22m7.815101678s" Sep 18 03:17:45.294316 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.294213 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:45.294469 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:45.294359 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:17:45.297253 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.297227 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:45.297372 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:45.297356 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:17:45.336427 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.336395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" event={"ID":"dddf8f39-e67b-48a2-bac4-2399e3718e57","Type":"ContainerStarted","Data":"fd76ffb8c9b817efd333796fec107f561767252f31e64c3a9fc179a58ff48bc4"} Sep 18 03:17:45.346262 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.346238 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d29f9" event={"ID":"b2814e15-58e8-4144-becf-e6af9314b295","Type":"ContainerStarted","Data":"f46e35bb4466090792d7b191dbe31b7f1d29ad1dd23398469b1a155ea7fb7760"} Sep 18 03:17:45.351006 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.350978 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" event={"ID":"578ed779-0fd5-4517-91b4-58b82bcc8a97","Type":"ContainerStarted","Data":"be5562444d6f1862ea1c4c7bf269369f0148d84ad7cef6658b300cbd4cef35ca"} Sep 18 03:17:45.355878 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.355854 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal" event={"ID":"c5d8fca0c5700e570250443bdef4a41a","Type":"ContainerStarted","Data":"019bbf7f46f6be9716fff2a2c6dfbe1231cc6c19f9f67e6f2ebabfe238214f6c"} Sep 18 03:17:45.363050 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.362922 2580 generic.go:356] "Generic (PLEG): container finished" podID="a44ec86bb6ecf4f75a2e225ac81e21a1" containerID="357ee03685df0b223030d906e93052b1c5fb53af5a75675c3a9cd742cd859bac" exitCode=0 Sep 18 03:17:45.363050 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.363005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" event={"ID":"a44ec86bb6ecf4f75a2e225ac81e21a1","Type":"ContainerDied","Data":"357ee03685df0b223030d906e93052b1c5fb53af5a75675c3a9cd742cd859bac"} Sep 18 03:17:45.377961 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.377908 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-90.ec2.internal" podStartSLOduration=3.377887585 podStartE2EDuration="3.377887585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:17:45.377283002 +0000 UTC m=+4.592724182" watchObservedRunningTime="2026-09-18 03:17:45.377887585 +0000 UTC m=+4.593328765" Sep 18 03:17:45.383164 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.383105 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b8sbn" event={"ID":"b3b737b9-7cab-470a-8c89-d20c07544375","Type":"ContainerStarted","Data":"cb87834925d3bcecee31e06c9284a1cb0626c99a1b0e72e3748bacc6b4cc1e34"} Sep 18 03:17:45.403605 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.403449 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fldjl" event={"ID":"48c50c68-f633-4e98-8823-587153372c93","Type":"ContainerStarted","Data":"7f02deff1fc730babe5e3cd4060d78e80eb752ae7a45d32a116e4e9d75a0b6c4"} Sep 18 03:17:45.415604 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.415504 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" event={"ID":"9c20f405-c42c-4c5b-b789-465af48ce967","Type":"ContainerStarted","Data":"8fffe558774fb7ed81fc6a5886ed39c1416e7144a8a86d9ebeadc40ad71d1d36"} Sep 18 03:17:45.427502 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.427474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerStarted","Data":"eefc28ca9bb2a19bb8fe1a6e9b71984dfb8bd642d55132cf275c390467cb5cab"} Sep 18 03:17:45.442856 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.442829 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t2nmd" event={"ID":"a1091cd1-b35a-49b0-b97c-a15ee4efbebe","Type":"ContainerStarted","Data":"c0e589457c104c4554960e13f30106a3ee08a535471c63dfbc3892abf9fce7fa"} Sep 18 03:17:45.460831 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:45.460758 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fdnrl" event={"ID":"5b477d64-4f58-4f2b-aca6-f898589820cb","Type":"ContainerStarted","Data":"18cf262278a4037e4bf95ed06b37cdf4ff447afe0a5e9fa0702411fd46671799"} Sep 18 03:17:46.293648 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:46.293614 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:46.294208 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:46.293739 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:17:46.485344 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:46.485160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" event={"ID":"a44ec86bb6ecf4f75a2e225ac81e21a1","Type":"ContainerStarted","Data":"a1aacd7cb208d87eecc2a412761bd4640ae5660b3f59a225dcf7378335fcf4c9"} Sep 18 03:17:46.505272 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:46.505220 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-90.ec2.internal" podStartSLOduration=4.505199528 podStartE2EDuration="4.505199528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:17:46.504897318 +0000 UTC m=+5.720338497" watchObservedRunningTime="2026-09-18 03:17:46.505199528 +0000 UTC m=+5.720640708" Sep 18 03:17:46.841430 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:46.841396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:46.841685 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:46.841472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:46.841685 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:46.841594 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:46.841685 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:46.841650 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs podName:ffd6a923-32d3-4212-8231-20b173a6a6a7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:50.84163281 +0000 UTC m=+10.057073965 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs") pod "network-metrics-daemon-ssqd2" (UID: "ffd6a923-32d3-4212-8231-20b173a6a6a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:46.842110 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:46.842089 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:46.842253 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:46.842166 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret podName:994a76dc-2129-4b42-8bb2-a51be4c091d7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:50.842150051 +0000 UTC m=+10.057591220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret") pod "global-pull-secret-syncer-fzpz8" (UID: "994a76dc-2129-4b42-8bb2-a51be4c091d7") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:46.942821 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:46.942788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:46.943022 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:46.942985 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:17:46.943022 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:46.943006 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:17:46.943022 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:46.943018 2580 projected.go:196] Error preparing data for projected volume kube-api-access-dk4tz for pod openshift-network-diagnostics/network-check-target-hwg2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:46.943255 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:46.943073 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz podName:f274a119-a4cd-4900-9ebc-4c3d7c023542 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:50.943056749 +0000 UTC m=+10.158497924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dk4tz" (UniqueName: "kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz") pod "network-check-target-hwg2m" (UID: "f274a119-a4cd-4900-9ebc-4c3d7c023542") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:47.295904 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:47.295209 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:47.295904 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:47.295370 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:17:47.295904 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:47.295412 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:47.295904 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:47.295526 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:17:48.294290 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:48.293669 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:48.294290 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:48.293845 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:17:49.298646 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:49.298609 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:49.299156 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:49.298738 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:17:49.299156 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:49.298609 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:49.299325 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:49.299276 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:17:50.293747 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:50.293689 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:50.293923 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:50.293825 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:17:50.876507 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:50.875816 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:50.876507 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:50.875872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:50.876507 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:50.875993 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:50.876507 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:50.876047 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret podName:994a76dc-2129-4b42-8bb2-a51be4c091d7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:58.876028709 +0000 UTC m=+18.091469879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret") pod "global-pull-secret-syncer-fzpz8" (UID: "994a76dc-2129-4b42-8bb2-a51be4c091d7") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:50.876507 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:50.876407 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:50.876507 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:50.876468 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs podName:ffd6a923-32d3-4212-8231-20b173a6a6a7 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:58.876452655 +0000 UTC m=+18.091893823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs") pod "network-metrics-daemon-ssqd2" (UID: "ffd6a923-32d3-4212-8231-20b173a6a6a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:50.977606 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:50.977077 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:50.977606 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:50.977215 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:17:50.977606 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:50.977238 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:17:50.977606 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:50.977252 2580 projected.go:196] Error preparing data for projected volume kube-api-access-dk4tz for pod openshift-network-diagnostics/network-check-target-hwg2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:50.977606 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:50.977306 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz podName:f274a119-a4cd-4900-9ebc-4c3d7c023542 nodeName:}" failed. No retries permitted until 2026-09-18 03:17:58.977288268 +0000 UTC m=+18.192729437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dk4tz" (UniqueName: "kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz") pod "network-check-target-hwg2m" (UID: "f274a119-a4cd-4900-9ebc-4c3d7c023542") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:51.296111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:51.293184 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:51.296111 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:51.293315 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:17:51.296111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:51.293659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:51.296111 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:51.293748 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:17:52.293161 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:52.293126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:52.293599 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:52.293234 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:17:53.293365 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:53.292868 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:53.293365 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:53.293003 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:17:53.294084 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:53.293928 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:53.294084 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:53.294034 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:17:54.293138 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:54.293108 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:54.293289 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:54.293204 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:17:55.293730 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:55.293696 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:55.294157 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:55.293824 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:17:55.294157 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:55.293886 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:55.294157 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:55.293973 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:17:56.292980 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:56.292950 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:56.293167 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:56.293083 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:17:57.293741 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:57.293709 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:57.294147 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:57.293752 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:57.294147 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:57.293857 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:17:57.294147 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:57.293973 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:17:58.292833 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:58.292805 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:58.293120 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:58.292917 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:17:58.940467 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:58.940434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:58.940867 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:58.940502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:58.940867 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:58.940603 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:58.940867 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:58.940607 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:58.940867 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:58.940659 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs podName:ffd6a923-32d3-4212-8231-20b173a6a6a7 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:14.940641673 +0000 UTC m=+34.156082838 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs") pod "network-metrics-daemon-ssqd2" (UID: "ffd6a923-32d3-4212-8231-20b173a6a6a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:17:58.940867 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:58.940677 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret podName:994a76dc-2129-4b42-8bb2-a51be4c091d7 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:14.940668235 +0000 UTC m=+34.156109390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret") pod "global-pull-secret-syncer-fzpz8" (UID: "994a76dc-2129-4b42-8bb2-a51be4c091d7") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:17:59.040888 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:59.040855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:17:59.041068 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:59.041013 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:17:59.041068 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:59.041033 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:17:59.041068 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:59.041045 2580 projected.go:196] Error preparing data for projected volume kube-api-access-dk4tz for pod openshift-network-diagnostics/network-check-target-hwg2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:59.041243 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:59.041120 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz podName:f274a119-a4cd-4900-9ebc-4c3d7c023542 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:15.04110144 +0000 UTC m=+34.256542614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dk4tz" (UniqueName: "kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz") pod "network-check-target-hwg2m" (UID: "f274a119-a4cd-4900-9ebc-4c3d7c023542") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:17:59.292825 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:59.292799 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:17:59.292944 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:59.292922 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:17:59.293004 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:17:59.292973 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:17:59.293086 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:17:59.293071 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:00.293144 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:00.293112 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:00.293533 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:00.293244 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:18:01.294193 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:01.294162 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:01.294613 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:01.294241 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:01.294613 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:01.294252 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:18:01.294613 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:01.294353 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:02.293334 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:02.293296 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:02.293492 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:02.293434 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:18:03.293104 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.292933 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:03.293680 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.292989 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:03.293680 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:03.293218 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:03.293680 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:03.293282 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:18:03.524029 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.523998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fdnrl" event={"ID":"5b477d64-4f58-4f2b-aca6-f898589820cb","Type":"ContainerStarted","Data":"21d9d8bb5615c5e4c88d99bb8b6ae345ea5e8823cef4be591f7d01084672005f"} Sep 18 03:18:03.526375 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.526348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" event={"ID":"dddf8f39-e67b-48a2-bac4-2399e3718e57","Type":"ContainerStarted","Data":"3d467e9e85e82736c0718e830498ff9704374115fc4276c8a20f60da6a08372b"} Sep 18 03:18:03.528119 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.528089 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d29f9" event={"ID":"b2814e15-58e8-4144-becf-e6af9314b295","Type":"ContainerStarted","Data":"f5ab34f51862e97cbf24473379cfa38b10608b31253735f736dfda6c5b9bfa25"} Sep 18 03:18:03.530376 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.530319 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" event={"ID":"578ed779-0fd5-4517-91b4-58b82bcc8a97","Type":"ContainerStarted","Data":"83a01f92108ecd168a1cf237f134bf54d9faafbf6f622654e18d493b00727b08"} Sep 18 03:18:03.532216 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.532192 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b8sbn" event={"ID":"b3b737b9-7cab-470a-8c89-d20c07544375","Type":"ContainerStarted","Data":"502d4be9e80f9e32552c85775523f2e9d1b544d94f38f45671905668aa8b6545"} Sep 18 03:18:03.534010 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.533760 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fldjl" event={"ID":"48c50c68-f633-4e98-8823-587153372c93","Type":"ContainerStarted","Data":"fabeec1f9c82feaa19cca51e618205c49ab0718d10d203ee06f2adf7d6968fc2"} Sep 18 03:18:03.537780 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.537599 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" event={"ID":"9c20f405-c42c-4c5b-b789-465af48ce967","Type":"ContainerStarted","Data":"703a28455f03be31a0a3774131cd6a4f3430ee23b617efe6083dec220bd5b52b"} Sep 18 03:18:03.537780 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.537658 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" event={"ID":"9c20f405-c42c-4c5b-b789-465af48ce967","Type":"ContainerStarted","Data":"803e66de89a41c8d09234d531a1abcab5275c446ca5fc713e0d36b8dc99d869e"} Sep 18 03:18:03.537780 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.537691 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" event={"ID":"9c20f405-c42c-4c5b-b789-465af48ce967","Type":"ContainerStarted","Data":"2fc1b409d6b9663ce1baa341517732fdc915567ed43bb45ab59f6d40a095e1b8"} Sep 18 03:18:03.537780 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.537720 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" event={"ID":"9c20f405-c42c-4c5b-b789-465af48ce967","Type":"ContainerStarted","Data":"439f28211c27ad0448ec3f7e12dacd8c70fca743a73c7f2ab6e695ca2f7aac1a"} Sep 18 03:18:03.537780 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.537748 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" event={"ID":"9c20f405-c42c-4c5b-b789-465af48ce967","Type":"ContainerStarted","Data":"50af4c91a98fe1a289d0e9d2197c26b7160b78bbf08c4d82db59b4831484ef16"} Sep 18 03:18:03.539738 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.539558 2580 generic.go:356] "Generic (PLEG): container finished" podID="7af14471-f55a-469b-a673-9f9903c399e3" containerID="7a8c37dd5b15b04004314556d9962d9aec151b65932b9abeb8c4ea8b8a17d96c" exitCode=0 Sep 18 03:18:03.539826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.539650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerDied","Data":"7a8c37dd5b15b04004314556d9962d9aec151b65932b9abeb8c4ea8b8a17d96c"} Sep 18 03:18:03.543736 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.543699 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fdnrl" podStartSLOduration=3.172548354 podStartE2EDuration="21.543686411s" totalImagesPullingTime="18.371138057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:03.543261296 +0000 UTC m=+22.758702472" watchObservedRunningTime="2026-09-18 03:18:03.543686411 +0000 UTC m=+22.759127587" Sep 18 03:18:03.576355 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.576304 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b8sbn" podStartSLOduration=6.716844323 podStartE2EDuration="21.576288149s" totalImagesPullingTime="14.859443826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:03.57620737 +0000 UTC m=+22.791648545" watchObservedRunningTime="2026-09-18 03:18:03.576288149 +0000 UTC m=+22.791729327" Sep 18 03:18:03.576877 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.576839 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d29f9" podStartSLOduration=3.201054897 podStartE2EDuration="21.576829468s" totalImagesPullingTime="18.375774571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:03.559758414 +0000 UTC m=+22.775199590" watchObservedRunningTime="2026-09-18 03:18:03.576829468 +0000 UTC m=+22.792270644" Sep 18 03:18:03.596157 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.596121 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fldjl" podStartSLOduration=4.160729818 podStartE2EDuration="22.596108416s" totalImagesPullingTime="18.435378598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:03.595481068 +0000 UTC m=+22.810922243" watchObservedRunningTime="2026-09-18 03:18:03.596108416 +0000 UTC m=+22.811549590" Sep 18 03:18:03.644330 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:03.644243 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pbf5f" podStartSLOduration=3.232531503 podStartE2EDuration="21.644230936s" totalImagesPullingTime="18.411699433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:03.619435084 +0000 UTC m=+22.834876259" watchObservedRunningTime="2026-09-18 03:18:03.644230936 +0000 UTC m=+22.859672112" Sep 18 03:18:04.222690 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:04.222663 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 18 03:18:04.256002 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:04.255920 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T03:18:04.222686121Z","UUID":"ba445c7b-c316-485f-842e-86b98f940f83","Handler":null,"Name":"","Endpoint":""} Sep 18 03:18:04.257368 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:04.257337 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 18 03:18:04.257464 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:04.257372 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 18 03:18:04.293503 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:04.293481 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:04.293871 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:04.293593 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:18:04.544572 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:04.544487 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t2nmd" event={"ID":"a1091cd1-b35a-49b0-b97c-a15ee4efbebe","Type":"ContainerStarted","Data":"6944b2709985a0a7144815596614409439b36ad00dd17bd05149fe99569d5cd4"} Sep 18 03:18:04.546709 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:04.546680 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" event={"ID":"578ed779-0fd5-4517-91b4-58b82bcc8a97","Type":"ContainerStarted","Data":"c76c3b35d34cd48797b86810654386dc0ba77032379090942a37c433bfad0ed2"} Sep 18 03:18:04.550354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:04.550310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" event={"ID":"9c20f405-c42c-4c5b-b789-465af48ce967","Type":"ContainerStarted","Data":"a67e29780ea5658b91a85fbf89356534950925d9967d81efbf9c122aa09234af"} Sep 18 03:18:04.561697 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:04.561647 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t2nmd" podStartSLOduration=5.187569955 podStartE2EDuration="23.561629016s" totalImagesPullingTime="18.374059061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:04.560876752 +0000 UTC m=+23.776317927" watchObservedRunningTime="2026-09-18 03:18:04.561629016 +0000 UTC m=+23.777070194" Sep 18 03:18:05.293099 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:05.293067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:05.293099 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:05.293094 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:05.293471 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:05.293189 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:18:05.293471 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:05.293319 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:05.555611 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:05.555540 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" event={"ID":"578ed779-0fd5-4517-91b4-58b82bcc8a97","Type":"ContainerStarted","Data":"6121f880b0525de98fc6d56c7e84bf3826144cdbbf2bb0dbe1afcc76b08b3a60"} Sep 18 03:18:06.293729 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:06.293700 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:06.294113 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:06.293803 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:18:06.561944 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:06.561900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" event={"ID":"9c20f405-c42c-4c5b-b789-465af48ce967","Type":"ContainerStarted","Data":"8f31df38351170384efd515e58f32738494bcc6c69ebfafe3c7da275d30e6ec6"} Sep 18 03:18:07.004664 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:07.004594 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:18:07.005545 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:07.005519 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:18:07.023088 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:07.023033 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tjvjh" podStartSLOduration=4.443419722 podStartE2EDuration="25.022968665s" totalImagesPullingTime="20.579548943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:05.576187602 +0000 UTC m=+24.791628777" watchObservedRunningTime="2026-09-18 03:18:07.022968665 +0000 UTC m=+26.238409839" Sep 18 03:18:07.293206 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:07.293138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:07.293353 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:07.293267 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:18:07.293353 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:07.293322 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:07.293467 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:07.293442 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:07.564203 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:07.564177 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:18:07.564787 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:07.564769 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b8sbn" Sep 18 03:18:08.292980 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.292809 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:08.293101 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:08.293070 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:18:08.570273 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.570246 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" event={"ID":"9c20f405-c42c-4c5b-b789-465af48ce967","Type":"ContainerStarted","Data":"d15b1d3fb614a9d9fb459a30cb3ec65ba7ede48e59c529a68446d632aeb91774"} Sep 18 03:18:08.571082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.570463 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:18:08.571082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.570533 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:18:08.571082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.570584 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:18:08.572912 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.572882 2580 generic.go:356] "Generic (PLEG): container finished" podID="7af14471-f55a-469b-a673-9f9903c399e3" containerID="4233ce45976ab102fb03e518f6982329577ea11a6382f0069e5e92ad8ca5f9f1" exitCode=0 Sep 18 03:18:08.573076 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.572953 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerDied","Data":"4233ce45976ab102fb03e518f6982329577ea11a6382f0069e5e92ad8ca5f9f1"} Sep 18 03:18:08.586753 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.586731 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:18:08.586914 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.586897 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:18:08.614561 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:08.614525 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" podStartSLOduration=9.140895692 podStartE2EDuration="27.614513351s" totalImagesPullingTime="18.473617659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:08.612347196 +0000 UTC m=+27.827788372" watchObservedRunningTime="2026-09-18 03:18:08.614513351 +0000 UTC m=+27.829954547" Sep 18 03:18:09.293269 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:09.293234 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:09.293531 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:09.293348 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:18:09.293531 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:09.293400 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:09.293531 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:09.293483 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:09.753416 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:09.753234 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fzpz8"] Sep 18 03:18:09.753686 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:09.753497 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:09.753686 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:09.753587 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:09.756801 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:09.756706 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hwg2m"] Sep 18 03:18:09.756934 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:09.756895 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:09.757068 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:09.757002 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:18:09.757793 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:09.757770 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ssqd2"] Sep 18 03:18:09.757892 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:09.757862 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:09.758027 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:09.758003 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:18:10.580227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:10.580192 2580 generic.go:356] "Generic (PLEG): container finished" podID="7af14471-f55a-469b-a673-9f9903c399e3" containerID="17bbfe579210d53393fcd51c625bda9ef97bf63511f2be780045cdbe399c84d3" exitCode=0 Sep 18 03:18:10.580384 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:10.580293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerDied","Data":"17bbfe579210d53393fcd51c625bda9ef97bf63511f2be780045cdbe399c84d3"} Sep 18 03:18:11.293842 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:11.293810 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:11.294302 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:11.293894 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:11.294302 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:11.293976 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:11.294302 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:11.294071 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:18:11.294302 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:11.294088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:11.294302 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:11.294196 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:18:11.585503 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:11.585475 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerStarted","Data":"2bf7e04303cdecd8c854093d1bbb4aec42a1814c4623e9799dc02ee93c11288b"} Sep 18 03:18:12.590319 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:12.590282 2580 generic.go:356] "Generic (PLEG): container finished" podID="7af14471-f55a-469b-a673-9f9903c399e3" containerID="2bf7e04303cdecd8c854093d1bbb4aec42a1814c4623e9799dc02ee93c11288b" exitCode=0 Sep 18 03:18:12.590779 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:12.590329 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerDied","Data":"2bf7e04303cdecd8c854093d1bbb4aec42a1814c4623e9799dc02ee93c11288b"} Sep 18 03:18:13.294096 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:13.292997 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:13.294096 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:13.293012 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:13.294096 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:13.293011 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:13.294096 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:13.293140 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:18:13.294096 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:13.293182 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:18:13.294096 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:13.293256 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:14.959877 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:14.959838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:14.960385 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:14.959885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:14.960385 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:14.959978 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 03:18:14.960385 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:14.959992 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:18:14.960385 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:14.960037 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret podName:994a76dc-2129-4b42-8bb2-a51be4c091d7 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:46.960021646 +0000 UTC m=+66.175462799 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret") pod "global-pull-secret-syncer-fzpz8" (UID: "994a76dc-2129-4b42-8bb2-a51be4c091d7") : object "kube-system"/"original-pull-secret" not registered Sep 18 03:18:14.960385 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:14.960069 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs podName:ffd6a923-32d3-4212-8231-20b173a6a6a7 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:46.960043954 +0000 UTC m=+66.175485107 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs") pod "network-metrics-daemon-ssqd2" (UID: "ffd6a923-32d3-4212-8231-20b173a6a6a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 03:18:15.060427 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:15.060397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:15.060588 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:15.060545 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 03:18:15.060588 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:15.060565 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 03:18:15.060588 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:15.060577 2580 projected.go:196] Error preparing data for projected volume kube-api-access-dk4tz for pod openshift-network-diagnostics/network-check-target-hwg2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:18:15.060751 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:15.060636 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz podName:f274a119-a4cd-4900-9ebc-4c3d7c023542 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:47.060617433 +0000 UTC m=+66.276058605 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dk4tz" (UniqueName: "kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz") pod "network-check-target-hwg2m" (UID: "f274a119-a4cd-4900-9ebc-4c3d7c023542") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 03:18:15.293300 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:15.293220 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:15.293443 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:15.293220 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:15.293443 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:15.293339 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hwg2m" podUID="f274a119-a4cd-4900-9ebc-4c3d7c023542" Sep 18 03:18:15.293530 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:15.293455 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssqd2" podUID="ffd6a923-32d3-4212-8231-20b173a6a6a7" Sep 18 03:18:15.293530 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:15.293220 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:15.293611 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:15.293550 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fzpz8" podUID="994a76dc-2129-4b42-8bb2-a51be4c091d7" Sep 18 03:18:16.457594 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.457427 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:16.567512 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.567486 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:16.603675 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.603651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerStarted","Data":"391f749498a5e74d3d05abd39fea6aba51191dd89ed64810541aa86b2cc85120"} Sep 18 03:18:16.624918 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.624898 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-90.ec2.internal" event="NodeReady" Sep 18 03:18:16.625076 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.625044 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 18 03:18:16.680700 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.680572 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5fqzd"] Sep 18 03:18:16.684460 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.684437 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-826hv"] Sep 18 03:18:16.684588 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.684569 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.687288 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.687267 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-826hv" Sep 18 03:18:16.687812 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.687794 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 18 03:18:16.687999 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.687794 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 18 03:18:16.687999 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.687946 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xc59\"" Sep 18 03:18:16.690228 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.690121 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 18 03:18:16.690228 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.690204 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 18 03:18:16.690615 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.690600 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ftrqs\"" Sep 18 03:18:16.690876 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.690859 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 18 03:18:16.692605 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.692587 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5fqzd"] Sep 18 03:18:16.696553 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.696533 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-826hv"] Sep 18 03:18:16.874015 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.873967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-tmp-dir\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.874015 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.874000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/991c83a8-e37c-4043-b60a-13c67bcca116-cert\") pod \"ingress-canary-826hv\" (UID: \"991c83a8-e37c-4043-b60a-13c67bcca116\") " pod="openshift-ingress-canary/ingress-canary-826hv" Sep 18 03:18:16.874153 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.874021 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-config-volume\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.874153 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.874046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwlvr\" (UniqueName: \"kubernetes.io/projected/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-kube-api-access-zwlvr\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.874153 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.874115 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr4bb\" (UniqueName: \"kubernetes.io/projected/991c83a8-e37c-4043-b60a-13c67bcca116-kube-api-access-zr4bb\") pod \"ingress-canary-826hv\" (UID: \"991c83a8-e37c-4043-b60a-13c67bcca116\") " pod="openshift-ingress-canary/ingress-canary-826hv" Sep 18 03:18:16.874242 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.874160 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.956428 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.956401 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:16.975406 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.975383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.975457 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.975426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-tmp-dir\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.975523 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:16.975509 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:18:16.975562 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.975537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/991c83a8-e37c-4043-b60a-13c67bcca116-cert\") pod \"ingress-canary-826hv\" (UID: \"991c83a8-e37c-4043-b60a-13c67bcca116\") " pod="openshift-ingress-canary/ingress-canary-826hv" Sep 18 03:18:16.975562 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:16.975555 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls podName:bcc9eec9-e533-4cd1-b3db-bed11c4b76c1 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:17.475541281 +0000 UTC m=+36.690982475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls") pod "dns-default-5fqzd" (UID: "bcc9eec9-e533-4cd1-b3db-bed11c4b76c1") : secret "dns-default-metrics-tls" not found Sep 18 03:18:16.975659 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.975585 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-config-volume\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.975659 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.975606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwlvr\" (UniqueName: \"kubernetes.io/projected/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-kube-api-access-zwlvr\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.975659 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.975626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr4bb\" (UniqueName: \"kubernetes.io/projected/991c83a8-e37c-4043-b60a-13c67bcca116-kube-api-access-zr4bb\") pod \"ingress-canary-826hv\" (UID: \"991c83a8-e37c-4043-b60a-13c67bcca116\") " pod="openshift-ingress-canary/ingress-canary-826hv" Sep 18 03:18:16.975785 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.975680 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-tmp-dir\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.976069 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.976021 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-config-volume\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.979776 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.979758 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/991c83a8-e37c-4043-b60a-13c67bcca116-cert\") pod \"ingress-canary-826hv\" (UID: \"991c83a8-e37c-4043-b60a-13c67bcca116\") " pod="openshift-ingress-canary/ingress-canary-826hv" Sep 18 03:18:16.986863 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.986841 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwlvr\" (UniqueName: \"kubernetes.io/projected/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-kube-api-access-zwlvr\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:16.986945 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:16.986871 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr4bb\" (UniqueName: \"kubernetes.io/projected/991c83a8-e37c-4043-b60a-13c67bcca116-kube-api-access-zr4bb\") pod \"ingress-canary-826hv\" (UID: \"991c83a8-e37c-4043-b60a-13c67bcca116\") " pod="openshift-ingress-canary/ingress-canary-826hv" Sep 18 03:18:17.042902 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.042883 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-826hv" Sep 18 03:18:17.056832 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.056814 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:17.296985 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.296925 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:17.297127 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.296927 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:17.297127 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.296930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:17.301854 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.301833 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 03:18:17.302000 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.301857 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 03:18:17.302000 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.301903 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 03:18:17.302000 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.301928 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n6f6t\"" Sep 18 03:18:17.302000 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.301928 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 03:18:17.302000 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.301983 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4b8hq\"" Sep 18 03:18:17.324278 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.324258 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-826hv"] Sep 18 03:18:17.328113 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:18:17.328091 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991c83a8_e37c_4043_b60a_13c67bcca116.slice/crio-b97871adf0ac2eccf8c9f7be1d6200fb078d1391632bf4242d16bec7c25f50a8 WatchSource:0}: Error finding container b97871adf0ac2eccf8c9f7be1d6200fb078d1391632bf4242d16bec7c25f50a8: Status 404 returned error can't find the container with id b97871adf0ac2eccf8c9f7be1d6200fb078d1391632bf4242d16bec7c25f50a8 Sep 18 03:18:17.479652 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.479628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:17.480326 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:17.479767 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:18:17.480326 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:17.479834 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls podName:bcc9eec9-e533-4cd1-b3db-bed11c4b76c1 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:18.479819422 +0000 UTC m=+37.695260602 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls") pod "dns-default-5fqzd" (UID: "bcc9eec9-e533-4cd1-b3db-bed11c4b76c1") : secret "dns-default-metrics-tls" not found Sep 18 03:18:17.607162 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.607136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-826hv" event={"ID":"991c83a8-e37c-4043-b60a-13c67bcca116","Type":"ContainerStarted","Data":"b97871adf0ac2eccf8c9f7be1d6200fb078d1391632bf4242d16bec7c25f50a8"} Sep 18 03:18:17.609953 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.609930 2580 generic.go:356] "Generic (PLEG): container finished" podID="7af14471-f55a-469b-a673-9f9903c399e3" containerID="391f749498a5e74d3d05abd39fea6aba51191dd89ed64810541aa86b2cc85120" exitCode=0 Sep 18 03:18:17.610066 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:17.609963 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerDied","Data":"391f749498a5e74d3d05abd39fea6aba51191dd89ed64810541aa86b2cc85120"} Sep 18 03:18:18.488000 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:18.487972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:18.488462 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:18.488117 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:18:18.488462 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:18.488200 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls podName:bcc9eec9-e533-4cd1-b3db-bed11c4b76c1 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:20.488180095 +0000 UTC m=+39.703621249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls") pod "dns-default-5fqzd" (UID: "bcc9eec9-e533-4cd1-b3db-bed11c4b76c1") : secret "dns-default-metrics-tls" not found Sep 18 03:18:18.617092 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:18.617033 2580 generic.go:356] "Generic (PLEG): container finished" podID="7af14471-f55a-469b-a673-9f9903c399e3" containerID="03eaee47964314483c926b02e085cef547f924123ab0c7a50787a0bf2a9e45db" exitCode=0 Sep 18 03:18:18.617266 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:18.617109 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerDied","Data":"03eaee47964314483c926b02e085cef547f924123ab0c7a50787a0bf2a9e45db"} Sep 18 03:18:18.919724 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:18.919691 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:19.020956 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:19.020886 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:19.622678 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:19.622646 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" event={"ID":"7af14471-f55a-469b-a673-9f9903c399e3","Type":"ContainerStarted","Data":"23f88fde257d5ba3d2281d5dfd3d53c896457415a48ca4a13477250e63a3c404"} Sep 18 03:18:19.652458 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:19.652415 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wvlk7" podStartSLOduration=9.087618067 podStartE2EDuration="37.652399226s" totalImagesPullingTime="28.564781159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:19.650531763 +0000 UTC m=+38.865972940" watchObservedRunningTime="2026-09-18 03:18:19.652399226 +0000 UTC m=+38.867840380" Sep 18 03:18:19.777606 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:19.777574 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:19.881624 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:19.881568 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:19.953398 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:19.953369 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:20.373960 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:20.373932 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:20.444877 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:20.444853 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:20.504882 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:20.504862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:20.505005 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:20.504993 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:18:20.505067 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:20.505041 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls podName:bcc9eec9-e533-4cd1-b3db-bed11c4b76c1 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:24.505029377 +0000 UTC m=+43.720470530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls") pod "dns-default-5fqzd" (UID: "bcc9eec9-e533-4cd1-b3db-bed11c4b76c1") : secret "dns-default-metrics-tls" not found Sep 18 03:18:20.625863 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:20.625809 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-826hv" event={"ID":"991c83a8-e37c-4043-b60a-13c67bcca116","Type":"ContainerStarted","Data":"2c996820ecaea8b9028807dc593f89e9dd4cc2a1d1a7daf398110fd013a2f0d0"} Sep 18 03:18:20.645333 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:20.645295 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-826hv" podStartSLOduration=1.97729782 podStartE2EDuration="4.645283216s" totalImagesPullingTime="2.667985396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:20.643884297 +0000 UTC m=+39.859325485" watchObservedRunningTime="2026-09-18 03:18:20.645283216 +0000 UTC m=+39.860724414" Sep 18 03:18:21.613695 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:21.613659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:21.709301 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:21.709271 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:24.531443 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:24.531405 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:24.531823 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:24.531534 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:18:24.531823 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:24.531590 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls podName:bcc9eec9-e533-4cd1-b3db-bed11c4b76c1 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:32.531575493 +0000 UTC m=+51.747016645 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls") pod "dns-default-5fqzd" (UID: "bcc9eec9-e533-4cd1-b3db-bed11c4b76c1") : secret "dns-default-metrics-tls" not found Sep 18 03:18:25.999338 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:25.999299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:26.185921 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:26.185889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:26.314214 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:26.314191 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:26.473429 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:26.473398 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:26.602438 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:26.602376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:26.920765 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:26.920704 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:26.992784 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:26.992754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:27.307197 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:27.307142 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:27.416513 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:27.416485 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:32.581703 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:32.581667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:32.582153 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:32.581812 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:18:32.582153 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:32.581885 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls podName:bcc9eec9-e533-4cd1-b3db-bed11c4b76c1 nodeName:}" failed. No retries permitted until 2026-09-18 03:18:48.581869267 +0000 UTC m=+67.797310420 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls") pod "dns-default-5fqzd" (UID: "bcc9eec9-e533-4cd1-b3db-bed11c4b76c1") : secret "dns-default-metrics-tls" not found Sep 18 03:18:37.950040 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:37.950006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:39.010358 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:39.010324 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:40.599967 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:40.599940 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m7g" Sep 18 03:18:40.799736 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:40.799704 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:40.901812 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:40.901739 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:41.000735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:41.000713 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:46.978473 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:46.978438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:46.978833 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:46.978490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:18:46.981702 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:46.981684 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 03:18:46.981829 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:46.981812 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 03:18:46.989187 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:46.989169 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 18 03:18:46.989261 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:46.989249 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs podName:ffd6a923-32d3-4212-8231-20b173a6a6a7 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:50.989228305 +0000 UTC m=+130.204669474 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs") pod "network-metrics-daemon-ssqd2" (UID: "ffd6a923-32d3-4212-8231-20b173a6a6a7") : secret "metrics-daemon-secret" not found Sep 18 03:18:46.992535 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:46.992519 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/994a76dc-2129-4b42-8bb2-a51be4c091d7-original-pull-secret\") pod \"global-pull-secret-syncer-fzpz8\" (UID: \"994a76dc-2129-4b42-8bb2-a51be4c091d7\") " pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:47.012871 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.012852 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fzpz8" Sep 18 03:18:47.079614 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.079577 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:47.083240 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.083221 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 03:18:47.093430 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.093408 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 03:18:47.103000 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.102973 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk4tz\" (UniqueName: \"kubernetes.io/projected/f274a119-a4cd-4900-9ebc-4c3d7c023542-kube-api-access-dk4tz\") pod \"network-check-target-hwg2m\" (UID: \"f274a119-a4cd-4900-9ebc-4c3d7c023542\") " pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:47.203778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.203751 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:47.284278 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.284222 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:47.320697 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.320677 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fzpz8"] Sep 18 03:18:47.321811 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.321793 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n6f6t\"" Sep 18 03:18:47.323957 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:18:47.323936 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod994a76dc_2129_4b42_8bb2_a51be4c091d7.slice/crio-bc3606d099efb0147730647f08d6534539ca01d422849bd6056c3eeb16962eff WatchSource:0}: Error finding container bc3606d099efb0147730647f08d6534539ca01d422849bd6056c3eeb16962eff: Status 404 returned error can't find the container with id bc3606d099efb0147730647f08d6534539ca01d422849bd6056c3eeb16962eff Sep 18 03:18:47.329111 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.329089 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:47.377153 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.377118 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:47.479016 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.478990 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:47.579300 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.579273 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:47.637985 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.637958 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hwg2m"] Sep 18 03:18:47.641994 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:18:47.641971 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf274a119_a4cd_4900_9ebc_4c3d7c023542.slice/crio-579e43c2b90cc1b36b1968b1827db7f36bfa0d8305525d4c233e4c6fef52c9b7 WatchSource:0}: Error finding container 579e43c2b90cc1b36b1968b1827db7f36bfa0d8305525d4c233e4c6fef52c9b7: Status 404 returned error can't find the container with id 579e43c2b90cc1b36b1968b1827db7f36bfa0d8305525d4c233e4c6fef52c9b7 Sep 18 03:18:47.690173 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.690146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fzpz8" event={"ID":"994a76dc-2129-4b42-8bb2-a51be4c091d7","Type":"ContainerStarted","Data":"bc3606d099efb0147730647f08d6534539ca01d422849bd6056c3eeb16962eff"} Sep 18 03:18:47.691278 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:47.691257 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hwg2m" event={"ID":"f274a119-a4cd-4900-9ebc-4c3d7c023542","Type":"ContainerStarted","Data":"579e43c2b90cc1b36b1968b1827db7f36bfa0d8305525d4c233e4c6fef52c9b7"} Sep 18 03:18:48.339543 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:48.339499 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:48.414625 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:48.414568 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:48.590389 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:48.590310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:18:48.590536 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:48.590439 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:18:48.590536 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:18:48.590504 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls podName:bcc9eec9-e533-4cd1-b3db-bed11c4b76c1 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:20.590484125 +0000 UTC m=+99.805925293 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls") pod "dns-default-5fqzd" (UID: "bcc9eec9-e533-4cd1-b3db-bed11c4b76c1") : secret "dns-default-metrics-tls" not found Sep 18 03:18:51.232881 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:51.232842 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:52.707770 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:52.707729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hwg2m" event={"ID":"f274a119-a4cd-4900-9ebc-4c3d7c023542","Type":"ContainerStarted","Data":"350f8ea26ad1354cadd3a9cf57e1f77f64df79f456ac0faf7834f8507ee554bf"} Sep 18 03:18:52.708280 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:52.707830 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:18:52.709584 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:52.709558 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fzpz8" event={"ID":"994a76dc-2129-4b42-8bb2-a51be4c091d7","Type":"ContainerStarted","Data":"1c910af6fa3ad4aa57697a418c6492d692a2ab18f1589f6c6f48e663f4ccd6aa"} Sep 18 03:18:52.726346 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:52.726293 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hwg2m" podStartSLOduration=67.153231683 podStartE2EDuration="1m11.726280025s" totalImagesPullingTime="4.573048342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:52.725662513 +0000 UTC m=+71.941103688" watchObservedRunningTime="2026-09-18 03:18:52.726280025 +0000 UTC m=+71.941721200" Sep 18 03:18:52.747510 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:52.745830 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fzpz8" podStartSLOduration=65.854829238 podStartE2EDuration="1m10.745808528s" totalImagesPullingTime="4.89097929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:18:52.743677093 +0000 UTC m=+71.959118268" watchObservedRunningTime="2026-09-18 03:18:52.745808528 +0000 UTC m=+71.961249705" Sep 18 03:18:55.219242 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:55.219209 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:59.476672 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:59.476626 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:18:59.549676 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:18:59.549642 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:04.359875 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:04.359741 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:04.461227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:04.461192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:04.560092 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:04.560046 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:12.966450 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:12.966411 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-vkr82"] Sep 18 03:19:12.970687 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:12.970668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:12.973813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:12.973781 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 18 03:19:12.973949 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:12.973815 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 18 03:19:12.973949 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:12.973878 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 18 03:19:12.975421 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:12.975383 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 18 03:19:12.975582 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:12.975404 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kkzlj\"" Sep 18 03:19:12.980472 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:12.980446 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 18 03:19:12.981580 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:12.981556 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-vkr82"] Sep 18 03:19:13.041007 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.040986 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff2a6591-33f5-4c87-99cf-969fa772ff61-serving-cert\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.041146 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.041027 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ff2a6591-33f5-4c87-99cf-969fa772ff61-snapshots\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.041146 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.041067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff2a6591-33f5-4c87-99cf-969fa772ff61-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.041146 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.041088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9mwx\" (UniqueName: \"kubernetes.io/projected/ff2a6591-33f5-4c87-99cf-969fa772ff61-kube-api-access-p9mwx\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.041146 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.041110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff2a6591-33f5-4c87-99cf-969fa772ff61-tmp\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.041146 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.041132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff2a6591-33f5-4c87-99cf-969fa772ff61-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.072431 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.072404 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6c9b9b47d6-s4lnj"] Sep 18 03:19:13.075185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.075168 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.079323 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.079304 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 18 03:19:13.080067 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.080027 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 18 03:19:13.080170 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.080099 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 18 03:19:13.080386 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.080368 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vkv89\"" Sep 18 03:19:13.080444 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.080368 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 18 03:19:13.080444 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.080409 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 18 03:19:13.083517 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.083500 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 18 03:19:13.091789 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.091769 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c9b9b47d6-s4lnj"] Sep 18 03:19:13.141902 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.141877 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-stats-auth\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.141998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.141907 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.141998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.141967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ff2a6591-33f5-4c87-99cf-969fa772ff61-snapshots\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.141998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.141995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdl48\" (UniqueName: \"kubernetes.io/projected/be3bf17c-370d-414e-891d-bb24a5768021-kube-api-access-rdl48\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.142160 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff2a6591-33f5-4c87-99cf-969fa772ff61-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.142160 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mwx\" (UniqueName: \"kubernetes.io/projected/ff2a6591-33f5-4c87-99cf-969fa772ff61-kube-api-access-p9mwx\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.142160 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-default-certificate\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.142160 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142105 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff2a6591-33f5-4c87-99cf-969fa772ff61-tmp\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.142160 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142130 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff2a6591-33f5-4c87-99cf-969fa772ff61-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.142401 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff2a6591-33f5-4c87-99cf-969fa772ff61-serving-cert\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.142401 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.142610 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142590 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff2a6591-33f5-4c87-99cf-969fa772ff61-tmp\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.142674 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142621 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ff2a6591-33f5-4c87-99cf-969fa772ff61-snapshots\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.142726 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142675 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff2a6591-33f5-4c87-99cf-969fa772ff61-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.142894 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.142877 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff2a6591-33f5-4c87-99cf-969fa772ff61-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.144474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.144451 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff2a6591-33f5-4c87-99cf-969fa772ff61-serving-cert\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.151847 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.151827 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9mwx\" (UniqueName: \"kubernetes.io/projected/ff2a6591-33f5-4c87-99cf-969fa772ff61-kube-api-access-p9mwx\") pod \"insights-operator-6f9c644dd9-vkr82\" (UID: \"ff2a6591-33f5-4c87-99cf-969fa772ff61\") " pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.178889 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.178869 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9"] Sep 18 03:19:13.181867 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.181854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.185933 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.185919 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 18 03:19:13.186140 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.186128 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 18 03:19:13.186399 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.186380 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 18 03:19:13.187067 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.187035 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-mbc6h\"" Sep 18 03:19:13.187688 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.187656 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 18 03:19:13.202659 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.202627 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-2j7sn"] Sep 18 03:19:13.205331 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.205317 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.209208 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.209190 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 18 03:19:13.209288 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.209196 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 18 03:19:13.210025 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.210006 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 18 03:19:13.210459 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.210439 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 18 03:19:13.211095 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.211078 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9"] Sep 18 03:19:13.215344 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.215325 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-c96xc\"" Sep 18 03:19:13.223240 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.223185 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 18 03:19:13.225081 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.225036 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-2j7sn"] Sep 18 03:19:13.243232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243212 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ac414f76-9056-4e31-a0d9-e09ffc8451a4-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-kwwd9\" (UID: \"ac414f76-9056-4e31-a0d9-e09ffc8451a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.243328 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243238 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d79bd980-6c95-4cf7-8f81-e53a08cc190e-serving-cert\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.243328 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.243328 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-stats-auth\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.243328 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.243525 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:13.243414 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:19:13.243525 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243448 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ac414f76-9056-4e31-a0d9-e09ffc8451a4-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-kwwd9\" (UID: \"ac414f76-9056-4e31-a0d9-e09ffc8451a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.243525 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d79bd980-6c95-4cf7-8f81-e53a08cc190e-trusted-ca\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.243525 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:13.243502 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:13.743483062 +0000 UTC m=+92.958924217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : secret "router-metrics-certs-default" not found Sep 18 03:19:13.243718 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:13.243529 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:13.743519942 +0000 UTC m=+92.958961094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : configmap references non-existent config key: service-ca.crt Sep 18 03:19:13.243718 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k88p4\" (UniqueName: \"kubernetes.io/projected/ac414f76-9056-4e31-a0d9-e09ffc8451a4-kube-api-access-k88p4\") pod \"kube-storage-version-migrator-operator-55889f7b47-kwwd9\" (UID: \"ac414f76-9056-4e31-a0d9-e09ffc8451a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.243718 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbx2h\" (UniqueName: \"kubernetes.io/projected/d79bd980-6c95-4cf7-8f81-e53a08cc190e-kube-api-access-kbx2h\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.243718 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdl48\" (UniqueName: \"kubernetes.io/projected/be3bf17c-370d-414e-891d-bb24a5768021-kube-api-access-rdl48\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.243718 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d79bd980-6c95-4cf7-8f81-e53a08cc190e-config\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.243973 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.243782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-default-certificate\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.245633 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.245614 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-stats-auth\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.245974 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.245960 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-default-certificate\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.253292 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.253271 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdl48\" (UniqueName: \"kubernetes.io/projected/be3bf17c-370d-414e-891d-bb24a5768021-kube-api-access-rdl48\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.280683 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.280659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" Sep 18 03:19:13.287837 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.287815 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd"] Sep 18 03:19:13.290787 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.290774 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd" Sep 18 03:19:13.293588 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.293564 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vhq9l\"" Sep 18 03:19:13.302237 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.302218 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd"] Sep 18 03:19:13.344580 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.344550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ac414f76-9056-4e31-a0d9-e09ffc8451a4-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-kwwd9\" (UID: \"ac414f76-9056-4e31-a0d9-e09ffc8451a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.344699 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.344596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d79bd980-6c95-4cf7-8f81-e53a08cc190e-serving-cert\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.344699 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.344656 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ac414f76-9056-4e31-a0d9-e09ffc8451a4-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-kwwd9\" (UID: \"ac414f76-9056-4e31-a0d9-e09ffc8451a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.344699 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.344682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d79bd980-6c95-4cf7-8f81-e53a08cc190e-trusted-ca\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.344856 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.344712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88p4\" (UniqueName: \"kubernetes.io/projected/ac414f76-9056-4e31-a0d9-e09ffc8451a4-kube-api-access-k88p4\") pod \"kube-storage-version-migrator-operator-55889f7b47-kwwd9\" (UID: \"ac414f76-9056-4e31-a0d9-e09ffc8451a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.344856 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.344754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbx2h\" (UniqueName: \"kubernetes.io/projected/d79bd980-6c95-4cf7-8f81-e53a08cc190e-kube-api-access-kbx2h\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.344856 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.344811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d79bd980-6c95-4cf7-8f81-e53a08cc190e-config\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.345010 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.344862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwwrc\" (UniqueName: \"kubernetes.io/projected/b53834df-51de-4d06-bc0e-24f87f33af0c-kube-api-access-kwwrc\") pod \"network-check-source-7cbc45fddc-rz2qd\" (UID: \"b53834df-51de-4d06-bc0e-24f87f33af0c\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd" Sep 18 03:19:13.345892 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.345650 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ac414f76-9056-4e31-a0d9-e09ffc8451a4-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-kwwd9\" (UID: \"ac414f76-9056-4e31-a0d9-e09ffc8451a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.345892 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.345845 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d79bd980-6c95-4cf7-8f81-e53a08cc190e-config\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.346072 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.345952 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d79bd980-6c95-4cf7-8f81-e53a08cc190e-trusted-ca\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.347435 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.347355 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d79bd980-6c95-4cf7-8f81-e53a08cc190e-serving-cert\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.347607 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.347585 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ac414f76-9056-4e31-a0d9-e09ffc8451a4-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-kwwd9\" (UID: \"ac414f76-9056-4e31-a0d9-e09ffc8451a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.353661 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.353641 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbx2h\" (UniqueName: \"kubernetes.io/projected/d79bd980-6c95-4cf7-8f81-e53a08cc190e-kube-api-access-kbx2h\") pod \"console-operator-fd4bb46c-2j7sn\" (UID: \"d79bd980-6c95-4cf7-8f81-e53a08cc190e\") " pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.355220 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.355200 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k88p4\" (UniqueName: \"kubernetes.io/projected/ac414f76-9056-4e31-a0d9-e09ffc8451a4-kube-api-access-k88p4\") pod \"kube-storage-version-migrator-operator-55889f7b47-kwwd9\" (UID: \"ac414f76-9056-4e31-a0d9-e09ffc8451a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.383619 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.383591 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:13.446082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.446030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwwrc\" (UniqueName: \"kubernetes.io/projected/b53834df-51de-4d06-bc0e-24f87f33af0c-kube-api-access-kwwrc\") pod \"network-check-source-7cbc45fddc-rz2qd\" (UID: \"b53834df-51de-4d06-bc0e-24f87f33af0c\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd" Sep 18 03:19:13.454449 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.454427 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwwrc\" (UniqueName: \"kubernetes.io/projected/b53834df-51de-4d06-bc0e-24f87f33af0c-kube-api-access-kwwrc\") pod \"network-check-source-7cbc45fddc-rz2qd\" (UID: \"b53834df-51de-4d06-bc0e-24f87f33af0c\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd" Sep 18 03:19:13.483952 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.483907 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:13.489866 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.489844 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" Sep 18 03:19:13.513726 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.513705 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:13.557705 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.557584 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-vkr82"] Sep 18 03:19:13.561208 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:13.561184 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff2a6591_33f5_4c87_99cf_969fa772ff61.slice/crio-703100bd5bd1d2f15113d6fd8d71b667f743faf103e01308af0cb5fb5b27412c WatchSource:0}: Error finding container 703100bd5bd1d2f15113d6fd8d71b667f743faf103e01308af0cb5fb5b27412c: Status 404 returned error can't find the container with id 703100bd5bd1d2f15113d6fd8d71b667f743faf103e01308af0cb5fb5b27412c Sep 18 03:19:13.599882 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.599860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd" Sep 18 03:19:13.749488 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.749429 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.749488 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.749468 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:13.749633 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:13.749595 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:19:13.749633 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:13.749627 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:14.749609274 +0000 UTC m=+93.965050447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : configmap references non-existent config key: service-ca.crt Sep 18 03:19:13.749703 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:13.749647 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:14.74963927 +0000 UTC m=+93.965080422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : secret "router-metrics-certs-default" not found Sep 18 03:19:13.761318 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.761292 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" event={"ID":"ff2a6591-33f5-4c87-99cf-969fa772ff61","Type":"ContainerStarted","Data":"703100bd5bd1d2f15113d6fd8d71b667f743faf103e01308af0cb5fb5b27412c"} Sep 18 03:19:13.761645 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.761592 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9"] Sep 18 03:19:13.764608 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:13.764588 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac414f76_9056_4e31_a0d9_e09ffc8451a4.slice/crio-f051f05805928697d4983e44f9dbd9bb71f778e857d819242b349724d2a22a93 WatchSource:0}: Error finding container f051f05805928697d4983e44f9dbd9bb71f778e857d819242b349724d2a22a93: Status 404 returned error can't find the container with id f051f05805928697d4983e44f9dbd9bb71f778e857d819242b349724d2a22a93 Sep 18 03:19:13.793402 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.793384 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-2j7sn"] Sep 18 03:19:13.795567 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:13.795551 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79bd980_6c95_4cf7_8f81_e53a08cc190e.slice/crio-eae8cc6a7889836a5d4096652221848f6e6c0fdedddccc0ac3c0f5c3df8897c0 WatchSource:0}: Error finding container eae8cc6a7889836a5d4096652221848f6e6c0fdedddccc0ac3c0f5c3df8897c0: Status 404 returned error can't find the container with id eae8cc6a7889836a5d4096652221848f6e6c0fdedddccc0ac3c0f5c3df8897c0 Sep 18 03:19:13.871553 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:13.871533 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd"] Sep 18 03:19:13.873728 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:13.873710 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb53834df_51de_4d06_bc0e_24f87f33af0c.slice/crio-9a7f083bc96687378d1605192e499c7fc8b041947daebadf79f417355364e6cb WatchSource:0}: Error finding container 9a7f083bc96687378d1605192e499c7fc8b041947daebadf79f417355364e6cb: Status 404 returned error can't find the container with id 9a7f083bc96687378d1605192e499c7fc8b041947daebadf79f417355364e6cb Sep 18 03:19:14.757541 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:14.757211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:14.757541 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:14.757258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:14.757541 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:14.757438 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:19:14.757541 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:14.757499 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:16.7574803 +0000 UTC m=+95.972921455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : secret "router-metrics-certs-default" not found Sep 18 03:19:14.757541 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:14.757518 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:16.757507476 +0000 UTC m=+95.972948632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : configmap references non-existent config key: service-ca.crt Sep 18 03:19:14.766794 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:14.766762 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" event={"ID":"d79bd980-6c95-4cf7-8f81-e53a08cc190e","Type":"ContainerStarted","Data":"eae8cc6a7889836a5d4096652221848f6e6c0fdedddccc0ac3c0f5c3df8897c0"} Sep 18 03:19:14.768406 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:14.768371 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" event={"ID":"ac414f76-9056-4e31-a0d9-e09ffc8451a4","Type":"ContainerStarted","Data":"f051f05805928697d4983e44f9dbd9bb71f778e857d819242b349724d2a22a93"} Sep 18 03:19:14.770480 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:14.770456 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd" event={"ID":"b53834df-51de-4d06-bc0e-24f87f33af0c","Type":"ContainerStarted","Data":"4407794f26024a9d858e185b07c40b5ef7317617f8c2bc1e09942e962ed7b0db"} Sep 18 03:19:14.770574 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:14.770500 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd" event={"ID":"b53834df-51de-4d06-bc0e-24f87f33af0c","Type":"ContainerStarted","Data":"9a7f083bc96687378d1605192e499c7fc8b041947daebadf79f417355364e6cb"} Sep 18 03:19:14.797337 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:14.797192 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rz2qd" podStartSLOduration=1.797173892 podStartE2EDuration="1.797173892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:14.796587153 +0000 UTC m=+94.012028328" watchObservedRunningTime="2026-09-18 03:19:14.797173892 +0000 UTC m=+94.012615066" Sep 18 03:19:15.390456 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:15.390422 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:15.493023 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:15.492984 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:16.730504 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.730479 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:16.774477 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.774448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:16.774582 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.774486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:16.774646 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:16.774596 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:19:16.774646 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:16.774610 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:20.77458762 +0000 UTC m=+99.990028786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : configmap references non-existent config key: service-ca.crt Sep 18 03:19:16.774742 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:16.774646 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:20.774631302 +0000 UTC m=+99.990072461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : secret "router-metrics-certs-default" not found Sep 18 03:19:16.778604 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.778578 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" event={"ID":"ff2a6591-33f5-4c87-99cf-969fa772ff61","Type":"ContainerStarted","Data":"cb0b61f06a8718484f98de1d2d22d906be4188555912582243279f481f251fb5"} Sep 18 03:19:16.780154 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.780124 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" event={"ID":"ac414f76-9056-4e31-a0d9-e09ffc8451a4","Type":"ContainerStarted","Data":"31e627e00b5e29cb466865de6cac782d05ae8f91746d61d284710617f42b0ce1"} Sep 18 03:19:16.781973 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.781954 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/0.log" Sep 18 03:19:16.782070 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.781994 2580 generic.go:356] "Generic (PLEG): container finished" podID="d79bd980-6c95-4cf7-8f81-e53a08cc190e" containerID="385032f53ce9e83d7f76a6b7dd95b9ce3b40db22f046a0b3224358db5bdb06ba" exitCode=255 Sep 18 03:19:16.782070 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.782029 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" event={"ID":"d79bd980-6c95-4cf7-8f81-e53a08cc190e","Type":"ContainerDied","Data":"385032f53ce9e83d7f76a6b7dd95b9ce3b40db22f046a0b3224358db5bdb06ba"} Sep 18 03:19:16.782264 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.782251 2580 scope.go:117] "RemoveContainer" containerID="385032f53ce9e83d7f76a6b7dd95b9ce3b40db22f046a0b3224358db5bdb06ba" Sep 18 03:19:16.802016 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.801975 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" podStartSLOduration=1.796361188 podStartE2EDuration="4.801959379s" totalImagesPullingTime="3.005598191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:16.799302337 +0000 UTC m=+96.014743513" watchObservedRunningTime="2026-09-18 03:19:16.801959379 +0000 UTC m=+96.017400555" Sep 18 03:19:16.805575 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.805555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:16.819632 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:16.818949 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" podStartSLOduration=1.012479398 podStartE2EDuration="3.818935822s" totalImagesPullingTime="2.806456424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:16.817693849 +0000 UTC m=+96.033135026" watchObservedRunningTime="2026-09-18 03:19:16.818935822 +0000 UTC m=+96.034377028" Sep 18 03:19:17.156367 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.156343 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:17.231833 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.231797 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:17.417239 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.417189 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:17.513370 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.513337 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:17.786707 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.786653 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/1.log" Sep 18 03:19:17.787144 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.787130 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/0.log" Sep 18 03:19:17.787212 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.787162 2580 generic.go:356] "Generic (PLEG): container finished" podID="d79bd980-6c95-4cf7-8f81-e53a08cc190e" containerID="ffdeeba2393903d8fdc359ed725f4320966380e918324318d29057d405b9c06c" exitCode=255 Sep 18 03:19:17.787265 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.787251 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" event={"ID":"d79bd980-6c95-4cf7-8f81-e53a08cc190e","Type":"ContainerDied","Data":"ffdeeba2393903d8fdc359ed725f4320966380e918324318d29057d405b9c06c"} Sep 18 03:19:17.787316 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.787304 2580 scope.go:117] "RemoveContainer" containerID="385032f53ce9e83d7f76a6b7dd95b9ce3b40db22f046a0b3224358db5bdb06ba" Sep 18 03:19:17.787475 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.787460 2580 scope.go:117] "RemoveContainer" containerID="ffdeeba2393903d8fdc359ed725f4320966380e918324318d29057d405b9c06c" Sep 18 03:19:17.787634 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:17.787616 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-2j7sn_openshift-console-operator(d79bd980-6c95-4cf7-8f81-e53a08cc190e)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" podUID="d79bd980-6c95-4cf7-8f81-e53a08cc190e" Sep 18 03:19:17.886624 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.886599 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:17.981276 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:17.981251 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:18.247132 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.247092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:18.268872 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.268844 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2hm5z"] Sep 18 03:19:18.275588 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.275566 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.278701 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.278682 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 18 03:19:18.278701 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.278700 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qzm8k\"" Sep 18 03:19:18.279215 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.279199 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 18 03:19:18.293998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.293977 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2hm5z"] Sep 18 03:19:18.343844 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.343825 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:18.387946 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.387922 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.388028 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.387985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.388028 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.388009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cf9a91b-ff80-4653-9046-1fbcc82cd395-data-volume\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.388133 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.388030 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7scl\" (UniqueName: \"kubernetes.io/projected/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-api-access-p7scl\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.388133 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.388078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0cf9a91b-ff80-4653-9046-1fbcc82cd395-crio-socket\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.489307 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.489283 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0cf9a91b-ff80-4653-9046-1fbcc82cd395-crio-socket\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.489307 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.489307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.489417 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.489354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.489417 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.489401 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0cf9a91b-ff80-4653-9046-1fbcc82cd395-crio-socket\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.489417 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:18.489411 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:19:18.489510 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:18.489422 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:19:18.489510 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:18.489459 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:18.989446199 +0000 UTC m=+98.204887353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : secret "insights-runtime-extractor-tls" not found Sep 18 03:19:18.489510 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.489454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cf9a91b-ff80-4653-9046-1fbcc82cd395-data-volume\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.489510 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:18.489471 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:18.989464309 +0000 UTC m=+98.204905462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : configmap "kube-rbac-proxy" not found Sep 18 03:19:18.489510 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.489493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7scl\" (UniqueName: \"kubernetes.io/projected/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-api-access-p7scl\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.489729 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.489714 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cf9a91b-ff80-4653-9046-1fbcc82cd395-data-volume\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.503954 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.503906 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7scl\" (UniqueName: \"kubernetes.io/projected/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-api-access-p7scl\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.791861 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.791787 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/1.log" Sep 18 03:19:18.792172 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.792118 2580 scope.go:117] "RemoveContainer" containerID="ffdeeba2393903d8fdc359ed725f4320966380e918324318d29057d405b9c06c" Sep 18 03:19:18.792285 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:18.792266 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-2j7sn_openshift-console-operator(d79bd980-6c95-4cf7-8f81-e53a08cc190e)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" podUID="d79bd980-6c95-4cf7-8f81-e53a08cc190e" Sep 18 03:19:18.936905 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.936877 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:18.992676 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.992651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.992764 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:18.992703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:18.992814 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:18.992762 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:19:18.992814 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:18.992778 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:19:18.992873 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:18.992815 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:19.992802302 +0000 UTC m=+99.208243454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : configmap "kube-rbac-proxy" not found Sep 18 03:19:18.992873 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:18.992832 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:19.992820553 +0000 UTC m=+99.208261706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : secret "insights-runtime-extractor-tls" not found Sep 18 03:19:19.017928 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.017904 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-2wgt5"] Sep 18 03:19:19.020982 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.020964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.024412 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.024388 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 18 03:19:19.024499 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.024391 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5d2zk\"" Sep 18 03:19:19.024499 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.024413 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 18 03:19:19.024741 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.024726 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 18 03:19:19.024815 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.024740 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 18 03:19:19.029943 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.029921 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-2wgt5"] Sep 18 03:19:19.037290 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.037270 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:19.092941 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.092920 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5eb2146a-c4b9-4e71-9b20-1ae16190c533-signing-key\") pod \"service-ca-55f6fc7b86-2wgt5\" (UID: \"5eb2146a-c4b9-4e71-9b20-1ae16190c533\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.093024 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.092949 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntn7q\" (UniqueName: \"kubernetes.io/projected/5eb2146a-c4b9-4e71-9b20-1ae16190c533-kube-api-access-ntn7q\") pod \"service-ca-55f6fc7b86-2wgt5\" (UID: \"5eb2146a-c4b9-4e71-9b20-1ae16190c533\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.093024 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.092972 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5eb2146a-c4b9-4e71-9b20-1ae16190c533-signing-cabundle\") pod \"service-ca-55f6fc7b86-2wgt5\" (UID: \"5eb2146a-c4b9-4e71-9b20-1ae16190c533\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.108878 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.108836 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:19.193884 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.193860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntn7q\" (UniqueName: \"kubernetes.io/projected/5eb2146a-c4b9-4e71-9b20-1ae16190c533-kube-api-access-ntn7q\") pod \"service-ca-55f6fc7b86-2wgt5\" (UID: \"5eb2146a-c4b9-4e71-9b20-1ae16190c533\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.193961 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.193894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5eb2146a-c4b9-4e71-9b20-1ae16190c533-signing-cabundle\") pod \"service-ca-55f6fc7b86-2wgt5\" (UID: \"5eb2146a-c4b9-4e71-9b20-1ae16190c533\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.194013 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.193980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5eb2146a-c4b9-4e71-9b20-1ae16190c533-signing-key\") pod \"service-ca-55f6fc7b86-2wgt5\" (UID: \"5eb2146a-c4b9-4e71-9b20-1ae16190c533\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.194564 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.194543 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5eb2146a-c4b9-4e71-9b20-1ae16190c533-signing-cabundle\") pod \"service-ca-55f6fc7b86-2wgt5\" (UID: \"5eb2146a-c4b9-4e71-9b20-1ae16190c533\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.196499 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.196478 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5eb2146a-c4b9-4e71-9b20-1ae16190c533-signing-key\") pod \"service-ca-55f6fc7b86-2wgt5\" (UID: \"5eb2146a-c4b9-4e71-9b20-1ae16190c533\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.203127 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.203107 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntn7q\" (UniqueName: \"kubernetes.io/projected/5eb2146a-c4b9-4e71-9b20-1ae16190c533-kube-api-access-ntn7q\") pod \"service-ca-55f6fc7b86-2wgt5\" (UID: \"5eb2146a-c4b9-4e71-9b20-1ae16190c533\") " pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.330186 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.330167 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" Sep 18 03:19:19.602781 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.602758 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-2wgt5"] Sep 18 03:19:19.604982 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:19.604953 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb2146a_c4b9_4e71_9b20_1ae16190c533.slice/crio-0e6a5abf68d1b7f9950a658bfaec9eb4ca7fb0abd54ff257703762e35b4ecebd WatchSource:0}: Error finding container 0e6a5abf68d1b7f9950a658bfaec9eb4ca7fb0abd54ff257703762e35b4ecebd: Status 404 returned error can't find the container with id 0e6a5abf68d1b7f9950a658bfaec9eb4ca7fb0abd54ff257703762e35b4ecebd Sep 18 03:19:19.799020 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.798953 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" event={"ID":"5eb2146a-c4b9-4e71-9b20-1ae16190c533","Type":"ContainerStarted","Data":"0e6a5abf68d1b7f9950a658bfaec9eb4ca7fb0abd54ff257703762e35b4ecebd"} Sep 18 03:19:19.999793 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.999755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:19.999888 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:19.999820 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:19.999923 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:19.999902 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:19:19.999923 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:19.999907 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:19:19.999998 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:19.999953 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:21.999938773 +0000 UTC m=+101.215379926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : configmap "kube-rbac-proxy" not found Sep 18 03:19:19.999998 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:19.999966 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:21.9999601 +0000 UTC m=+101.215401253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : secret "insights-runtime-extractor-tls" not found Sep 18 03:19:20.093918 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:20.093887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:20.154466 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:20.154446 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d29f9_b2814e15-58e8-4144-becf-e6af9314b295/dns-node-resolver/0.log" Sep 18 03:19:20.604581 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:20.604556 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:19:20.604749 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:20.604685 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 03:19:20.604749 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:20.604741 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls podName:bcc9eec9-e533-4cd1-b3db-bed11c4b76c1 nodeName:}" failed. No retries permitted until 2026-09-18 03:20:24.604722917 +0000 UTC m=+163.820164072 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls") pod "dns-default-5fqzd" (UID: "bcc9eec9-e533-4cd1-b3db-bed11c4b76c1") : secret "dns-default-metrics-tls" not found Sep 18 03:19:20.805031 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:20.805005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:20.805387 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:20.805039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:20.805387 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:20.805166 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:28.805149365 +0000 UTC m=+108.020590519 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : configmap references non-existent config key: service-ca.crt Sep 18 03:19:20.805387 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:20.805208 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 18 03:19:20.805387 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:20.805255 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:28.805243154 +0000 UTC m=+108.020684307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : secret "router-metrics-certs-default" not found Sep 18 03:19:20.951952 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:20.951887 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fdnrl_5b477d64-4f58-4f2b-aca6-f898589820cb/node-ca/0.log" Sep 18 03:19:21.757820 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:21.757766 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-826hv_991c83a8-e37c-4043-b60a-13c67bcca116/serve-healthcheck-canary/0.log" Sep 18 03:19:21.806842 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:21.806814 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" event={"ID":"5eb2146a-c4b9-4e71-9b20-1ae16190c533","Type":"ContainerStarted","Data":"4623ddbe3c568ab4f380c13d38e8ef7a8c1aafd28d320a0d91230f78a0f15d8b"} Sep 18 03:19:21.831551 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:21.831472 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-2wgt5" podStartSLOduration=1.87271583 podStartE2EDuration="3.831452797s" totalImagesPullingTime="1.958736967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:21.830928629 +0000 UTC m=+101.046369804" watchObservedRunningTime="2026-09-18 03:19:21.831452797 +0000 UTC m=+101.046893973" Sep 18 03:19:22.015091 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:22.015040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:22.015187 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:22.015151 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:19:22.015237 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:22.015209 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:26.015195994 +0000 UTC m=+105.230637160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : secret "insights-runtime-extractor-tls" not found Sep 18 03:19:22.015237 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:22.015148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:22.015332 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:22.015207 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:19:22.015366 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:22.015351 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:26.015332063 +0000 UTC m=+105.230773217 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : configmap "kube-rbac-proxy" not found Sep 18 03:19:23.135664 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:23.135615 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:23.461628 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:23.461535 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:23.514384 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:23.514358 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:23.514506 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:23.514415 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:23.514740 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:23.514726 2580 scope.go:117] "RemoveContainer" containerID="ffdeeba2393903d8fdc359ed725f4320966380e918324318d29057d405b9c06c" Sep 18 03:19:23.514899 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:23.514879 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-2j7sn_openshift-console-operator(d79bd980-6c95-4cf7-8f81-e53a08cc190e)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" podUID="d79bd980-6c95-4cf7-8f81-e53a08cc190e" Sep 18 03:19:23.591113 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:23.591074 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:23.715069 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:23.714993 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hwg2m" Sep 18 03:19:23.825655 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:23.825626 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:23.980746 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:23.980680 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:24.163332 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:24.163305 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:24.611983 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:24.611961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:25.245440 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:25.245410 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:25.413044 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:25.413020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:25.609314 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:25.609291 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:25.806362 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:25.806339 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:26.046714 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:26.046667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:26.046811 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:26.046728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:26.046811 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:26.046791 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 03:19:26.046890 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:26.046843 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 03:19:26.046890 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:26.046848 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:34.046834694 +0000 UTC m=+113.262275847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : configmap "kube-rbac-proxy" not found Sep 18 03:19:26.046890 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:26.046879 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls podName:0cf9a91b-ff80-4653-9046-1fbcc82cd395 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:34.046869202 +0000 UTC m=+113.262310354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2hm5z" (UID: "0cf9a91b-ff80-4653-9046-1fbcc82cd395") : secret "insights-runtime-extractor-tls" not found Sep 18 03:19:26.641360 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:26.641340 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:26.739877 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:26.739858 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:27.001129 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:27.001078 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:27.075644 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:27.075623 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:27.439643 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:27.439624 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:27.573037 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:27.573011 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:27.652664 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:27.652639 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:28.000970 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:28.000945 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:28.793989 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:28.793964 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:28.866839 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:28.866814 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:28.866960 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:28.866852 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:28.866998 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:28.866961 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle podName:be3bf17c-370d-414e-891d-bb24a5768021 nodeName:}" failed. No retries permitted until 2026-09-18 03:19:44.866947074 +0000 UTC m=+124.082388227 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle") pod "router-default-6c9b9b47d6-s4lnj" (UID: "be3bf17c-370d-414e-891d-bb24a5768021") : configmap references non-existent config key: service-ca.crt Sep 18 03:19:28.869119 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:28.869103 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be3bf17c-370d-414e-891d-bb24a5768021-metrics-certs\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:29.004596 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:29.004570 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:29.193323 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:29.193297 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:29.577893 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:29.577870 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:29.649170 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:29.649144 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:30.200083 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:30.200045 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:30.377128 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:30.377100 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:30.593632 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:30.593608 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:31.002030 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:31.001973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:31.451721 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:31.451694 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:31.722842 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:31.722782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:31.850726 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:31.850705 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:32.040123 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:32.040076 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:32.656121 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:32.656092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:33.020966 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:33.020900 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:33.402440 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:33.402405 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:33.627809 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:33.627782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:33.829762 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:33.829734 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:34.103789 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:34.103724 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:34.103922 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:34.103789 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:34.104317 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:34.104297 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cf9a91b-ff80-4653-9046-1fbcc82cd395-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:34.106588 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:34.106567 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cf9a91b-ff80-4653-9046-1fbcc82cd395-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2hm5z\" (UID: \"0cf9a91b-ff80-4653-9046-1fbcc82cd395\") " pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:34.184191 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:34.184163 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2hm5z" Sep 18 03:19:34.469961 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:34.469904 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2hm5z"] Sep 18 03:19:34.473235 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:34.473211 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf9a91b_ff80_4653_9046_1fbcc82cd395.slice/crio-2a9320e0f593bcedc553fb6adc73681a07619e29f7871ec6a85aaea675f3881d WatchSource:0}: Error finding container 2a9320e0f593bcedc553fb6adc73681a07619e29f7871ec6a85aaea675f3881d: Status 404 returned error can't find the container with id 2a9320e0f593bcedc553fb6adc73681a07619e29f7871ec6a85aaea675f3881d Sep 18 03:19:34.848005 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:34.847971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2hm5z" event={"ID":"0cf9a91b-ff80-4653-9046-1fbcc82cd395","Type":"ContainerStarted","Data":"11fd2a68e443eed0fedbb0877292750f0c1e860b0f1fa640fb5e7904ed922cbd"} Sep 18 03:19:34.848351 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:34.848026 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2hm5z" event={"ID":"0cf9a91b-ff80-4653-9046-1fbcc82cd395","Type":"ContainerStarted","Data":"2a9320e0f593bcedc553fb6adc73681a07619e29f7871ec6a85aaea675f3881d"} Sep 18 03:19:35.293648 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:35.293629 2580 scope.go:117] "RemoveContainer" containerID="ffdeeba2393903d8fdc359ed725f4320966380e918324318d29057d405b9c06c" Sep 18 03:19:35.853615 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:35.853581 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/2.log" Sep 18 03:19:35.854075 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:35.854039 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/1.log" Sep 18 03:19:35.854132 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:35.854093 2580 generic.go:356] "Generic (PLEG): container finished" podID="d79bd980-6c95-4cf7-8f81-e53a08cc190e" containerID="823bbb2a8baf5b3a9940b3e800c1a260cfeab7e168460da00fe4a53eef7948b3" exitCode=255 Sep 18 03:19:35.854197 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:35.854173 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" event={"ID":"d79bd980-6c95-4cf7-8f81-e53a08cc190e","Type":"ContainerDied","Data":"823bbb2a8baf5b3a9940b3e800c1a260cfeab7e168460da00fe4a53eef7948b3"} Sep 18 03:19:35.854282 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:35.854251 2580 scope.go:117] "RemoveContainer" containerID="ffdeeba2393903d8fdc359ed725f4320966380e918324318d29057d405b9c06c" Sep 18 03:19:35.854565 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:35.854540 2580 scope.go:117] "RemoveContainer" containerID="823bbb2a8baf5b3a9940b3e800c1a260cfeab7e168460da00fe4a53eef7948b3" Sep 18 03:19:35.854735 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:35.854713 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-fd4bb46c-2j7sn_openshift-console-operator(d79bd980-6c95-4cf7-8f81-e53a08cc190e)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" podUID="d79bd980-6c95-4cf7-8f81-e53a08cc190e" Sep 18 03:19:35.856329 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:35.856310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2hm5z" event={"ID":"0cf9a91b-ff80-4653-9046-1fbcc82cd395","Type":"ContainerStarted","Data":"235c4647e5b14356d86557972e71fe4bb610293796871a83a17817f7b690fecf"} Sep 18 03:19:36.596471 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:36.596439 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:36.694214 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:36.694191 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:36.860488 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:36.860440 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/2.log" Sep 18 03:19:36.862650 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:36.862628 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2hm5z" event={"ID":"0cf9a91b-ff80-4653-9046-1fbcc82cd395","Type":"ContainerStarted","Data":"6fc10cd1f24adc170e03587bea5a39a2142ede9652d26b8c1296b51a4ad0b05e"} Sep 18 03:19:36.887255 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:36.887211 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2hm5z" podStartSLOduration=16.88222829 podStartE2EDuration="18.887198055s" totalImagesPullingTime="2.004969765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:19:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:36.884963183 +0000 UTC m=+116.100404374" watchObservedRunningTime="2026-09-18 03:19:36.887198055 +0000 UTC m=+116.102639229" Sep 18 03:19:42.913767 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:42.913734 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-94655745f-w4r6z"] Sep 18 03:19:42.916870 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:42.916854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:42.921512 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:42.921494 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 18 03:19:42.921621 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:42.921565 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 18 03:19:42.923011 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:42.922991 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-btm9v\"" Sep 18 03:19:42.923011 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:42.923005 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 18 03:19:42.938883 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:42.938858 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 18 03:19:42.939742 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:42.939716 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-94655745f-w4r6z"] Sep 18 03:19:42.942260 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:42.942239 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:43.065332 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.065308 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b994e472-f9f7-405f-9fa8-d441eb56646b-installation-pull-secrets\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.065461 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.065335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b994e472-f9f7-405f-9fa8-d441eb56646b-registry-certificates\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.065461 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.065364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b994e472-f9f7-405f-9fa8-d441eb56646b-trusted-ca\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.065461 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.065380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b994e472-f9f7-405f-9fa8-d441eb56646b-bound-sa-token\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.065461 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.065398 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b994e472-f9f7-405f-9fa8-d441eb56646b-registry-tls\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.065461 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.065428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frbvl\" (UniqueName: \"kubernetes.io/projected/b994e472-f9f7-405f-9fa8-d441eb56646b-kube-api-access-frbvl\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.065609 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.065474 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b994e472-f9f7-405f-9fa8-d441eb56646b-image-registry-private-configuration\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.065609 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.065525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b994e472-f9f7-405f-9fa8-d441eb56646b-ca-trust-extracted\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.117721 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.117700 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:43.166762 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.166717 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b994e472-f9f7-405f-9fa8-d441eb56646b-installation-pull-secrets\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.166762 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.166740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b994e472-f9f7-405f-9fa8-d441eb56646b-registry-certificates\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.166872 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.166767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b994e472-f9f7-405f-9fa8-d441eb56646b-trusted-ca\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.166872 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.166784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b994e472-f9f7-405f-9fa8-d441eb56646b-bound-sa-token\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.166872 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.166799 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b994e472-f9f7-405f-9fa8-d441eb56646b-registry-tls\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.167007 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.166924 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frbvl\" (UniqueName: \"kubernetes.io/projected/b994e472-f9f7-405f-9fa8-d441eb56646b-kube-api-access-frbvl\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.167007 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.166972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b994e472-f9f7-405f-9fa8-d441eb56646b-image-registry-private-configuration\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.167126 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.167028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b994e472-f9f7-405f-9fa8-d441eb56646b-ca-trust-extracted\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.167372 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.167348 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b994e472-f9f7-405f-9fa8-d441eb56646b-ca-trust-extracted\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.167600 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.167584 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b994e472-f9f7-405f-9fa8-d441eb56646b-registry-certificates\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.168103 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.168083 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b994e472-f9f7-405f-9fa8-d441eb56646b-trusted-ca\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.169412 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.169380 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b994e472-f9f7-405f-9fa8-d441eb56646b-installation-pull-secrets\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.169850 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.169829 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b994e472-f9f7-405f-9fa8-d441eb56646b-image-registry-private-configuration\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.170027 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.170009 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b994e472-f9f7-405f-9fa8-d441eb56646b-registry-tls\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.181412 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.181392 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b994e472-f9f7-405f-9fa8-d441eb56646b-bound-sa-token\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.188693 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.188672 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frbvl\" (UniqueName: \"kubernetes.io/projected/b994e472-f9f7-405f-9fa8-d441eb56646b-kube-api-access-frbvl\") pod \"image-registry-94655745f-w4r6z\" (UID: \"b994e472-f9f7-405f-9fa8-d441eb56646b\") " pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.224935 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.224911 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.380938 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.380909 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:43.512155 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.512103 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-94655745f-w4r6z"] Sep 18 03:19:43.513998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.513974 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:43.514102 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.514012 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:43.514325 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.514311 2580 scope.go:117] "RemoveContainer" containerID="823bbb2a8baf5b3a9940b3e800c1a260cfeab7e168460da00fe4a53eef7948b3" Sep 18 03:19:43.514476 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:19:43.514458 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-fd4bb46c-2j7sn_openshift-console-operator(d79bd980-6c95-4cf7-8f81-e53a08cc190e)\"" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" podUID="d79bd980-6c95-4cf7-8f81-e53a08cc190e" Sep 18 03:19:43.514924 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:43.514901 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb994e472_f9f7_405f_9fa8_d441eb56646b.slice/crio-e1b3c32bd7136ff876764db94ac45f2c84981a8a6a2e5efaf36380aa866e9b21 WatchSource:0}: Error finding container e1b3c32bd7136ff876764db94ac45f2c84981a8a6a2e5efaf36380aa866e9b21: Status 404 returned error can't find the container with id e1b3c32bd7136ff876764db94ac45f2c84981a8a6a2e5efaf36380aa866e9b21 Sep 18 03:19:43.563148 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.563122 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:43.886193 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.886166 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94655745f-w4r6z" event={"ID":"b994e472-f9f7-405f-9fa8-d441eb56646b","Type":"ContainerStarted","Data":"8c7276fa8f5c5882f1425457ab48ca60d061401aca49e86351e421d982db255b"} Sep 18 03:19:43.886477 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.886211 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94655745f-w4r6z" event={"ID":"b994e472-f9f7-405f-9fa8-d441eb56646b","Type":"ContainerStarted","Data":"e1b3c32bd7136ff876764db94ac45f2c84981a8a6a2e5efaf36380aa866e9b21"} Sep 18 03:19:43.886477 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.886272 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:19:43.919829 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:43.919785 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-94655745f-w4r6z" podStartSLOduration=1.91976953 podStartE2EDuration="1.91976953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:43.91919289 +0000 UTC m=+123.134634074" watchObservedRunningTime="2026-09-18 03:19:43.91976953 +0000 UTC m=+123.135210734" Sep 18 03:19:44.574430 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:44.574404 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:44.685406 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:44.685381 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:44.879739 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:44.879688 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:44.880233 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:44.880214 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3bf17c-370d-414e-891d-bb24a5768021-service-ca-bundle\") pod \"router-default-6c9b9b47d6-s4lnj\" (UID: \"be3bf17c-370d-414e-891d-bb24a5768021\") " pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:44.887557 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:44.887538 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vkv89\"" Sep 18 03:19:44.893858 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:44.893834 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:45.187735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:45.187686 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c9b9b47d6-s4lnj"] Sep 18 03:19:45.190472 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:45.190451 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3bf17c_370d_414e_891d_bb24a5768021.slice/crio-a6d32f92d41cf108797a06b20e127adcf5320c6f9e6c0f7845b324e529949495 WatchSource:0}: Error finding container a6d32f92d41cf108797a06b20e127adcf5320c6f9e6c0f7845b324e529949495: Status 404 returned error can't find the container with id a6d32f92d41cf108797a06b20e127adcf5320c6f9e6c0f7845b324e529949495 Sep 18 03:19:45.894692 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:45.894663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" event={"ID":"be3bf17c-370d-414e-891d-bb24a5768021","Type":"ContainerStarted","Data":"8bce80f9e5529c121211e99ab62fa9ed9c5805cadcca797c3359b2dd5c15ffeb"} Sep 18 03:19:45.894849 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:45.894704 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" event={"ID":"be3bf17c-370d-414e-891d-bb24a5768021","Type":"ContainerStarted","Data":"a6d32f92d41cf108797a06b20e127adcf5320c6f9e6c0f7845b324e529949495"} Sep 18 03:19:45.930101 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:45.930032 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" podStartSLOduration=32.930017303 podStartE2EDuration="32.930017303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:45.927991564 +0000 UTC m=+125.143432737" watchObservedRunningTime="2026-09-18 03:19:45.930017303 +0000 UTC m=+125.145458477" Sep 18 03:19:46.894369 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:46.894342 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:46.896591 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:46.896573 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:46.898843 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:46.898825 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:46.899778 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:46.899760 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6c9b9b47d6-s4lnj" Sep 18 03:19:47.562873 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:47.562848 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:47.808157 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:47.808126 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:47.915021 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:47.914943 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:50.282106 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:50.282074 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:50.350474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:50.350445 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:50.873364 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:50.873339 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:50.968862 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:50.968834 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:51.019530 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:51.019505 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:19:51.027454 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:51.027426 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6a923-32d3-4212-8231-20b173a6a6a7-metrics-certs\") pod \"network-metrics-daemon-ssqd2\" (UID: \"ffd6a923-32d3-4212-8231-20b173a6a6a7\") " pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:19:51.195298 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:51.195246 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:51.210275 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:51.210255 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4b8hq\"" Sep 18 03:19:51.217997 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:51.217978 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssqd2" Sep 18 03:19:51.519562 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:51.519539 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ssqd2"] Sep 18 03:19:51.521973 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:51.521949 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd6a923_32d3_4212_8231_20b173a6a6a7.slice/crio-ba57af2fbe22feb2d0b007955b3431741b5eb74496749cbbafd977487a14e39f WatchSource:0}: Error finding container ba57af2fbe22feb2d0b007955b3431741b5eb74496749cbbafd977487a14e39f: Status 404 returned error can't find the container with id ba57af2fbe22feb2d0b007955b3431741b5eb74496749cbbafd977487a14e39f Sep 18 03:19:51.914589 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:51.914567 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ssqd2" event={"ID":"ffd6a923-32d3-4212-8231-20b173a6a6a7","Type":"ContainerStarted","Data":"ba57af2fbe22feb2d0b007955b3431741b5eb74496749cbbafd977487a14e39f"} Sep 18 03:19:52.921370 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:52.921342 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ssqd2" event={"ID":"ffd6a923-32d3-4212-8231-20b173a6a6a7","Type":"ContainerStarted","Data":"66e499abfc9e5ab8758515320e32ca4649a14d5e6c30209b13f84d09279fe6cf"} Sep 18 03:19:52.921688 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:52.921388 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ssqd2" event={"ID":"ffd6a923-32d3-4212-8231-20b173a6a6a7","Type":"ContainerStarted","Data":"74fb71404f9196f7e24b9c7898ffdbf2b5efe99857c242deb4bf37d2a87a13bc"} Sep 18 03:19:52.942822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:52.942775 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ssqd2" podStartSLOduration=129.980376439 podStartE2EDuration="2m10.94276131s" totalImagesPullingTime="962.384871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:52.941507364 +0000 UTC m=+132.156948562" watchObservedRunningTime="2026-09-18 03:19:52.94276131 +0000 UTC m=+132.158202509" Sep 18 03:19:53.078983 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:53.078962 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:53.168563 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:53.168542 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:53.619136 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:53.619109 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:54.678523 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:54.678496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:54.769192 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:54.769169 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:56.103624 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.103575 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:56.338450 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.338400 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:56.359904 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.359840 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fhfrj"] Sep 18 03:19:56.363546 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.363529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.368515 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.368495 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 18 03:19:56.371070 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.369683 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 18 03:19:56.371070 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.369711 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 18 03:19:56.371070 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.369784 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 18 03:19:56.371070 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.369974 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 18 03:19:56.371070 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.370217 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rhbqh\"" Sep 18 03:19:56.371070 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.370377 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 18 03:19:56.499560 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.499530 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-lp7d7"] Sep 18 03:19:56.502185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.502162 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:56.502761 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.502747 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.505951 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.505932 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 18 03:19:56.506217 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.506195 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 18 03:19:56.506491 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.506477 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 18 03:19:56.506557 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.506493 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mwln2\"" Sep 18 03:19:56.516102 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.516080 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-lp7d7"] Sep 18 03:19:56.555876 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.555856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.555958 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.555886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b86d4755-ee15-4b98-87b9-589f06a77272-metrics-client-ca\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.555958 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.555910 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nrhn\" (UniqueName: \"kubernetes.io/projected/b86d4755-ee15-4b98-87b9-589f06a77272-kube-api-access-4nrhn\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.555958 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.555939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-accelerators-collector-config\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.556113 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.555965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-tls\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.556113 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.555983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b86d4755-ee15-4b98-87b9-589f06a77272-sys\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.556113 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.555998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b86d4755-ee15-4b98-87b9-589f06a77272-root\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.556113 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.556032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-wtmp\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.556291 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.556142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-textfile\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.590254 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.590233 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:56.657155 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpdxd\" (UniqueName: \"kubernetes.io/projected/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-api-access-dpdxd\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.657155 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.657300 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-textfile\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657300 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657300 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b86d4755-ee15-4b98-87b9-589f06a77272-metrics-client-ca\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657300 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.657496 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657307 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.657496 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657343 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nrhn\" (UniqueName: \"kubernetes.io/projected/b86d4755-ee15-4b98-87b9-589f06a77272-kube-api-access-4nrhn\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657496 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.657496 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657427 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-textfile\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657496 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657471 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-accelerators-collector-config\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657738 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-tls\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657738 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657547 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b86d4755-ee15-4b98-87b9-589f06a77272-sys\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657738 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b86d4755-ee15-4b98-87b9-589f06a77272-root\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657738 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657618 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.657738 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657624 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b86d4755-ee15-4b98-87b9-589f06a77272-root\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657738 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657643 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b86d4755-ee15-4b98-87b9-589f06a77272-sys\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.657738 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-wtmp\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.658030 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657751 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-wtmp\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.658030 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.657834 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b86d4755-ee15-4b98-87b9-589f06a77272-metrics-client-ca\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.658198 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.658177 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-accelerators-collector-config\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.659822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.659794 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.659896 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.659873 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b86d4755-ee15-4b98-87b9-589f06a77272-node-exporter-tls\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.666845 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.666828 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nrhn\" (UniqueName: \"kubernetes.io/projected/b86d4755-ee15-4b98-87b9-589f06a77272-kube-api-access-4nrhn\") pod \"node-exporter-fhfrj\" (UID: \"b86d4755-ee15-4b98-87b9-589f06a77272\") " pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.674813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.674793 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fhfrj" Sep 18 03:19:56.685853 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:56.685836 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb86d4755_ee15_4b98_87b9_589f06a77272.slice/crio-6e4d8646eb19e14d7fd74bcdb88d8dc50d9901ff14dba4f25fa7bfc7694596de WatchSource:0}: Error finding container 6e4d8646eb19e14d7fd74bcdb88d8dc50d9901ff14dba4f25fa7bfc7694596de: Status 404 returned error can't find the container with id 6e4d8646eb19e14d7fd74bcdb88d8dc50d9901ff14dba4f25fa7bfc7694596de Sep 18 03:19:56.690068 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.690009 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:56.758476 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.758453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.758557 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.758478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.758557 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.758498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.758557 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.758521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.758557 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.758543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpdxd\" (UniqueName: \"kubernetes.io/projected/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-api-access-dpdxd\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.758557 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.758558 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.758955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.758937 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.759547 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.759525 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.759631 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.759615 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.760680 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.760663 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.760882 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.760865 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.775598 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.775576 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpdxd\" (UniqueName: \"kubernetes.io/projected/a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2-kube-api-access-dpdxd\") pod \"kube-state-metrics-84c885658c-lp7d7\" (UID: \"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.791790 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.791771 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:56.811335 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.811311 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" Sep 18 03:19:56.889459 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.889366 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:56.935933 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:56.935880 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fhfrj" event={"ID":"b86d4755-ee15-4b98-87b9-589f06a77272","Type":"ContainerStarted","Data":"6e4d8646eb19e14d7fd74bcdb88d8dc50d9901ff14dba4f25fa7bfc7694596de"} Sep 18 03:19:57.099404 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.099377 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-lp7d7"] Sep 18 03:19:57.103462 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:57.103438 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b0ae5d_3fbd_414b_9cfa_235a9276f6b2.slice/crio-4de1f3ecfa9571f3b65a7cd91069447f6e69e2e7bd82c8eff9061fc6ec6636b3 WatchSource:0}: Error finding container 4de1f3ecfa9571f3b65a7cd91069447f6e69e2e7bd82c8eff9061fc6ec6636b3: Status 404 returned error can't find the container with id 4de1f3ecfa9571f3b65a7cd91069447f6e69e2e7bd82c8eff9061fc6ec6636b3 Sep 18 03:19:57.293433 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.293380 2580 scope.go:117] "RemoveContainer" containerID="823bbb2a8baf5b3a9940b3e800c1a260cfeab7e168460da00fe4a53eef7948b3" Sep 18 03:19:57.333716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.333688 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:57.438090 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.438042 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:57.530549 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.530506 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 18 03:19:57.534886 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.534864 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.538341 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.538315 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:57.539282 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.539259 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 18 03:19:57.539393 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.539364 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 18 03:19:57.539471 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.539394 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 18 03:19:57.539528 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.539507 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 18 03:19:57.539616 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.539600 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2ghjq\"" Sep 18 03:19:57.541456 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.541389 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 18 03:19:57.542422 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.542400 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 18 03:19:57.542559 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.542520 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 18 03:19:57.542764 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.542747 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 18 03:19:57.555588 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.555566 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 18 03:19:57.557699 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.557672 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 18 03:19:57.636170 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.636124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:57.666673 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.666647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.666807 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.666676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.666807 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.666704 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.666807 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.666781 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.666970 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.666820 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.666970 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.666881 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.666970 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.666928 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-web-config\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.666970 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.666952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.667152 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.666987 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.667152 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.667065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r9qk\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-kube-api-access-6r9qk\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.667152 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.667098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-config-out\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.667152 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.667119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.667152 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.667140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.767725 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.767700 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.767834 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.767747 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.767834 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.767788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.767834 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.767817 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-web-config\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.767998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.767843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.767998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.767879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.767998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.767918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r9qk\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-kube-api-access-6r9qk\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.767998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.767948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-config-out\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.767998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.767979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.768258 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.768008 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.768258 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.768081 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.768258 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.768108 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.768258 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.768137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.769746 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.769716 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.770599 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.768082 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.771375 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.771301 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.771473 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.771392 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-web-config\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.771553 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.771535 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.771871 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.771839 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.772445 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.772391 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.772553 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.772511 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.773471 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.773437 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.773567 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.773468 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-config-out\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.773918 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.773899 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.774498 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.774476 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.783918 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.783896 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r9qk\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-kube-api-access-6r9qk\") pod \"alertmanager-main-0\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.847166 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.847138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:19:57.947846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.946951 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/2.log" Sep 18 03:19:57.947846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.947116 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" event={"ID":"d79bd980-6c95-4cf7-8f81-e53a08cc190e","Type":"ContainerStarted","Data":"b25184a40a9eb1fa61bde48c4558e569573ad34fdfbe4c7b19dbc2468b01ddd3"} Sep 18 03:19:57.947846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.947739 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:57.950578 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.950549 2580 generic.go:356] "Generic (PLEG): container finished" podID="b86d4755-ee15-4b98-87b9-589f06a77272" containerID="78703b3b968da59c0132e70ffad09188a14f546e369c39bad5d0d31efd18129c" exitCode=0 Sep 18 03:19:57.950688 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.950658 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fhfrj" event={"ID":"b86d4755-ee15-4b98-87b9-589f06a77272","Type":"ContainerDied","Data":"78703b3b968da59c0132e70ffad09188a14f546e369c39bad5d0d31efd18129c"} Sep 18 03:19:57.955723 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.955656 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" event={"ID":"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2","Type":"ContainerStarted","Data":"4de1f3ecfa9571f3b65a7cd91069447f6e69e2e7bd82c8eff9061fc6ec6636b3"} Sep 18 03:19:57.957155 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.957114 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" Sep 18 03:19:57.973321 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:57.973274 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-2j7sn" podStartSLOduration=42.19531523 podStartE2EDuration="44.973260828s" totalImagesPullingTime="2.777945598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:57.972594931 +0000 UTC m=+137.188036133" watchObservedRunningTime="2026-09-18 03:19:57.973260828 +0000 UTC m=+137.188702003" Sep 18 03:19:58.165950 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.165867 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:58.182576 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:58.182548 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf53fd542_85b5_466c_96b7_558b737918b2.slice/crio-0bb63f02c4ddb8e46f021f4ffbb861483bdf250e2df8c546660a5abc4d6299f0 WatchSource:0}: Error finding container 0bb63f02c4ddb8e46f021f4ffbb861483bdf250e2df8c546660a5abc4d6299f0: Status 404 returned error can't find the container with id 0bb63f02c4ddb8e46f021f4ffbb861483bdf250e2df8c546660a5abc4d6299f0 Sep 18 03:19:58.183097 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.183071 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 18 03:19:58.357288 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.357254 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:58.423755 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.423669 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-58cdb4884b-m92cq"] Sep 18 03:19:58.428017 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.427993 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.433998 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.433972 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rqj7d\"" Sep 18 03:19:58.434150 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.434115 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8br31gp2r05bq\"" Sep 18 03:19:58.435535 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.435330 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 18 03:19:58.435535 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.435415 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 18 03:19:58.435535 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.435428 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 18 03:19:58.435535 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.435336 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 18 03:19:58.435942 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.435621 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 18 03:19:58.456831 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.456802 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-58cdb4884b-m92cq"] Sep 18 03:19:58.458586 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.458554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:58.576384 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.576207 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/944d25f1-d00f-4e5f-864b-0621a45388ef-metrics-client-ca\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.576384 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.576293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.576384 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.576340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-tls\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.576652 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.576439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.576652 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.576518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-grpc-tls\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.576652 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.576544 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxvnc\" (UniqueName: \"kubernetes.io/projected/944d25f1-d00f-4e5f-864b-0621a45388ef-kube-api-access-rxvnc\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.576652 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.576573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.576652 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.576628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.677970 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.677918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-tls\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.677970 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.677958 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.678098 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.677998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-grpc-tls\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.678098 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.678022 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxvnc\" (UniqueName: \"kubernetes.io/projected/944d25f1-d00f-4e5f-864b-0621a45388ef-kube-api-access-rxvnc\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.678098 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.678074 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.678210 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.678108 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.678210 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.678140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/944d25f1-d00f-4e5f-864b-0621a45388ef-metrics-client-ca\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.678210 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.678190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.678896 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.678868 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/944d25f1-d00f-4e5f-864b-0621a45388ef-metrics-client-ca\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.680498 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.680478 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.680963 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.680940 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.681076 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.680959 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-tls\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.681125 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.681094 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.681203 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.681187 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.681237 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.681213 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/944d25f1-d00f-4e5f-864b-0621a45388ef-secret-grpc-tls\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.689733 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.689711 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxvnc\" (UniqueName: \"kubernetes.io/projected/944d25f1-d00f-4e5f-864b-0621a45388ef-kube-api-access-rxvnc\") pod \"thanos-querier-58cdb4884b-m92cq\" (UID: \"944d25f1-d00f-4e5f-864b-0621a45388ef\") " pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.738659 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.738640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:19:58.839748 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.839708 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:58.964407 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.964300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:58.966987 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.966945 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" event={"ID":"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2","Type":"ContainerStarted","Data":"2914165ef90a17a2cfa9dc9c4aecc72addc48064f380555beb3dfab0fce96487"} Sep 18 03:19:58.967185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.967010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" event={"ID":"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2","Type":"ContainerStarted","Data":"3b1eb4aca8841d4b98d511b6ddc7a92cb50f2d94acc4f7c6f9e7ddd37fa00f7d"} Sep 18 03:19:58.967185 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.967041 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" event={"ID":"a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2","Type":"ContainerStarted","Data":"ce43286fd57bab56dc94a523aed5218863d4ce381911a5f7512be8261fb89ef5"} Sep 18 03:19:58.969306 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.969277 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerStarted","Data":"0bb63f02c4ddb8e46f021f4ffbb861483bdf250e2df8c546660a5abc4d6299f0"} Sep 18 03:19:58.973603 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.973579 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fhfrj" event={"ID":"b86d4755-ee15-4b98-87b9-589f06a77272","Type":"ContainerStarted","Data":"7ad8026b396489ba047edf9df868dc00b352fc429607513cf7d6536f5e4a13d4"} Sep 18 03:19:58.973710 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:58.973628 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fhfrj" event={"ID":"b86d4755-ee15-4b98-87b9-589f06a77272","Type":"ContainerStarted","Data":"e4c339c01fe15d1a675587f3c42249492e09af0a812c9de079701dc8add524cc"} Sep 18 03:19:59.001414 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.001218 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-lp7d7" podStartSLOduration=1.453650351 podStartE2EDuration="3.001199461s" totalImagesPullingTime="1.54754911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:58.999040213 +0000 UTC m=+138.214481389" watchObservedRunningTime="2026-09-18 03:19:59.001199461 +0000 UTC m=+138.216640637" Sep 18 03:19:59.031685 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.031316 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fhfrj" podStartSLOduration=1.9274228230000001 podStartE2EDuration="3.031297818s" totalImagesPullingTime="1.030284744s" totalInitContainerRuntime="73.590251ms" isStatefulPod=true podCreationTimestamp="2026-09-18 03:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:19:59.029938393 +0000 UTC m=+138.245379578" watchObservedRunningTime="2026-09-18 03:19:59.031297818 +0000 UTC m=+138.246738994" Sep 18 03:19:59.039699 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.039662 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-58cdb4884b-m92cq"] Sep 18 03:19:59.041748 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:19:59.041721 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944d25f1_d00f_4e5f_864b_0621a45388ef.slice/crio-ec257574cc0e2a882c29a422ada8396d866640bc76d1cbf8d24cf165480ca08a WatchSource:0}: Error finding container ec257574cc0e2a882c29a422ada8396d866640bc76d1cbf8d24cf165480ca08a: Status 404 returned error can't find the container with id ec257574cc0e2a882c29a422ada8396d866640bc76d1cbf8d24cf165480ca08a Sep 18 03:19:59.191584 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.191548 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:59.521627 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.521590 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:59.628618 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.628587 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:59.793011 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.792946 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:59.892495 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.892459 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:19:59.979855 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.979646 2580 generic.go:356] "Generic (PLEG): container finished" podID="f53fd542-85b5-466c-96b7-558b737918b2" containerID="f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7" exitCode=0 Sep 18 03:19:59.979855 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.979745 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerDied","Data":"f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7"} Sep 18 03:19:59.983024 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:19:59.982766 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" event={"ID":"944d25f1-d00f-4e5f-864b-0621a45388ef","Type":"ContainerStarted","Data":"ec257574cc0e2a882c29a422ada8396d866640bc76d1cbf8d24cf165480ca08a"} Sep 18 03:20:00.040068 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:00.040025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:00.325959 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:00.325926 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:00.437616 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:00.437578 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:01.145521 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.145481 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:01.241597 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.241570 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:01.451017 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.450987 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:01.713679 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.713655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:01.994565 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.994463 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerStarted","Data":"52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27"} Sep 18 03:20:01.994752 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.994563 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerStarted","Data":"0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650"} Sep 18 03:20:01.994752 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.994599 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerStarted","Data":"dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677"} Sep 18 03:20:01.994752 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.994623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerStarted","Data":"178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901"} Sep 18 03:20:01.994752 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.994643 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerStarted","Data":"256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706"} Sep 18 03:20:01.996716 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.996684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" event={"ID":"944d25f1-d00f-4e5f-864b-0621a45388ef","Type":"ContainerStarted","Data":"a9e9ac72a9dd921122509e815a67a999dea81810d92cceac79ef4fb0b7c1de54"} Sep 18 03:20:01.996816 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.996747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" event={"ID":"944d25f1-d00f-4e5f-864b-0621a45388ef","Type":"ContainerStarted","Data":"bf20ae85cfa5e5dd302f314eb0aee3de31ee5a239d90bf35677f1568cb584c56"} Sep 18 03:20:01.996816 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:01.996772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" event={"ID":"944d25f1-d00f-4e5f-864b-0621a45388ef","Type":"ContainerStarted","Data":"244009c70b6a4fb9ea8799a35d0bf6a8f9dd60f73880e807827ec0afa4318871"} Sep 18 03:20:03.003725 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.003651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" event={"ID":"944d25f1-d00f-4e5f-864b-0621a45388ef","Type":"ContainerStarted","Data":"7588bcac1c03aca789f31de49725a7a2522f74213c3dd6c86efc8bb8fc7af26a"} Sep 18 03:20:03.004115 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.003731 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" event={"ID":"944d25f1-d00f-4e5f-864b-0621a45388ef","Type":"ContainerStarted","Data":"f332c3ec8dfbff4641a4c14a328e2abdf4ce7281e509d0dba684e489cc0c04d0"} Sep 18 03:20:03.004115 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.003775 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:20:03.004115 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.003812 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" event={"ID":"944d25f1-d00f-4e5f-864b-0621a45388ef","Type":"ContainerStarted","Data":"5022a6b25fa7d73b5d00adcfb9c2f0deb2263b3797bf79efd23ecb5e71fa2234"} Sep 18 03:20:03.006961 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.006938 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerStarted","Data":"4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931"} Sep 18 03:20:03.057597 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.057549 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" podStartSLOduration=1.983798652 podStartE2EDuration="5.057534205s" totalImagesPullingTime="3.073735553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:20:03.056954958 +0000 UTC m=+142.272396133" watchObservedRunningTime="2026-09-18 03:20:03.057534205 +0000 UTC m=+142.272975740" Sep 18 03:20:03.089184 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.089156 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:03.090582 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.090563 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:20:03.094381 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.094366 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.104861 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.104843 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 18 03:20:03.104988 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.104970 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 18 03:20:03.105082 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.104986 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 18 03:20:03.105358 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.105344 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 18 03:20:03.105413 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.105367 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 18 03:20:03.105413 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.105390 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 18 03:20:03.115625 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.115608 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.115703 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.115637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.115703 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.115658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-config-out\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.115703 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.115696 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.115876 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.115744 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.115876 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.115775 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvtc6\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-kube-api-access-fvtc6\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.115876 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.115802 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.115876 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.115829 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116090 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116021 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 18 03:20:03.116090 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116021 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116143 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 18 03:20:03.116227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116163 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116227 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116187 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-web-config\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116469 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116469 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116250 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116469 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116354 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116469 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-config\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116469 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116462 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116746 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116746 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116518 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 18 03:20:03.116746 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116543 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 18 03:20:03.116746 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.116746 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.116587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.117476 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.117458 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-coagstnrr4qo8\"" Sep 18 03:20:03.117570 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.117536 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 18 03:20:03.117570 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.117549 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 18 03:20:03.117945 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.117919 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 18 03:20:03.124543 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.124449 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pnn5x\"" Sep 18 03:20:03.126653 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.126631 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 18 03:20:03.127578 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.127561 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:20:03.128407 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.128364 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.522637593 podStartE2EDuration="6.128328333s" totalImagesPullingTime="3.589325187s" totalInitContainerRuntime="16.365553ms" isStatefulPod=false podCreationTimestamp="2026-09-18 03:19:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:20:03.127858199 +0000 UTC m=+142.343299373" watchObservedRunningTime="2026-09-18 03:20:03.128328333 +0000 UTC m=+142.343769507" Sep 18 03:20:03.131336 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.131128 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 18 03:20:03.217115 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217262 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217262 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217142 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217262 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217173 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217262 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-web-config\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217262 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217262 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217219 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217262 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217262 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217264 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-config\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217344 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217371 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217395 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217497 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-config-out\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217535 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvtc6\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-kube-api-access-fvtc6\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.217642 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.218190 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217827 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.218190 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.217931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.218190 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.218116 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.218345 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.218297 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.218400 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.218370 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.219559 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.219503 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.219794 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.219775 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.220930 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.220886 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-web-config\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.222238 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.221768 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.222238 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.221863 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.222238 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.221872 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.222238 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.222198 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-config-out\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.222735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.222717 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.222943 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.222920 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.223206 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.223187 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.223332 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.223316 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.223390 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.223348 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-config\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.223429 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.223414 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.223583 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.223568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.234522 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.234502 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvtc6\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-kube-api-access-fvtc6\") pod \"prometheus-k8s-0\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.353481 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.353463 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:03.403635 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.403616 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:03.739015 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.738994 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:20:03.740972 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:20:03.740946 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b538f8_2611_4e7b_a389_e52e771368e1.slice/crio-34a5caf940f5a1779350d04dabef7db68fd92fc732939b9f45fade58454248f6 WatchSource:0}: Error finding container 34a5caf940f5a1779350d04dabef7db68fd92fc732939b9f45fade58454248f6: Status 404 returned error can't find the container with id 34a5caf940f5a1779350d04dabef7db68fd92fc732939b9f45fade58454248f6 Sep 18 03:20:03.839964 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.839939 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:03.978404 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:03.978368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:04.011747 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:04.011700 2580 generic.go:356] "Generic (PLEG): container finished" podID="65b538f8-2611-4e7b-a389-e52e771368e1" containerID="f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3" exitCode=0 Sep 18 03:20:04.011999 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:04.011778 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerDied","Data":"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3"} Sep 18 03:20:04.011999 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:04.011842 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerStarted","Data":"34a5caf940f5a1779350d04dabef7db68fd92fc732939b9f45fade58454248f6"} Sep 18 03:20:04.894470 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:04.894442 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-94655745f-w4r6z" Sep 18 03:20:05.163417 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:05.163347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:05.278069 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:05.278021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:06.658793 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:06.658762 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:06.774191 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:06.774156 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:07.649725 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:07.649697 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:07.859199 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:07.859173 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:08.034810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:08.034783 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerStarted","Data":"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef"} Sep 18 03:20:08.034937 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:08.034841 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerStarted","Data":"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3"} Sep 18 03:20:08.034937 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:08.034877 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerStarted","Data":"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31"} Sep 18 03:20:08.034937 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:08.034915 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerStarted","Data":"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477"} Sep 18 03:20:08.035079 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:08.034950 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerStarted","Data":"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88"} Sep 18 03:20:08.035079 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:08.034991 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerStarted","Data":"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19"} Sep 18 03:20:08.123950 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:08.123858 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.8205079130000001 podStartE2EDuration="5.123841365s" totalImagesPullingTime="3.281073565s" totalInitContainerRuntime="22.259887ms" isStatefulPod=false podCreationTimestamp="2026-09-18 03:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:20:08.120193221 +0000 UTC m=+147.335634395" watchObservedRunningTime="2026-09-18 03:20:08.123841365 +0000 UTC m=+147.339282541" Sep 18 03:20:08.404621 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:08.404561 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:20:09.017866 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:09.017841 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-58cdb4884b-m92cq" Sep 18 03:20:09.853324 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:09.853296 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:09.980842 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:09.980811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:13.074401 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:13.074373 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:13.190709 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:13.190678 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:16.209190 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:16.209150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:16.313347 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:16.313318 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:17.269959 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:17.269923 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:17.399457 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:17.399426 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:17.610550 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:17.610510 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:17.766012 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:17.765967 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:17.891474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:17.891380 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:19.737950 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:20:19.737905 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5fqzd" podUID="bcc9eec9-e533-4cd1-b3db-bed11c4b76c1" Sep 18 03:20:20.083973 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:20.083944 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5fqzd" Sep 18 03:20:21.297537 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:21.297506 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:21.449219 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:21.449192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:21.682143 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:21.682120 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:21.781654 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:21.781627 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:23.065249 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:23.065204 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:23.164673 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:23.164644 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:24.693295 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:24.693254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:20:24.695694 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:24.695671 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bcc9eec9-e533-4cd1-b3db-bed11c4b76c1-metrics-tls\") pod \"dns-default-5fqzd\" (UID: \"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1\") " pod="openshift-dns/dns-default-5fqzd" Sep 18 03:20:24.889046 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:24.889019 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xc59\"" Sep 18 03:20:24.894691 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:24.894659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5fqzd" Sep 18 03:20:25.184495 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:25.184472 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5fqzd"] Sep 18 03:20:25.186591 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:20:25.186566 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc9eec9_e533_4cd1_b3db_bed11c4b76c1.slice/crio-17d337144bd7a40f315f63bc62d9f8183222c6fdf199a59cf27fb23a67d6b002 WatchSource:0}: Error finding container 17d337144bd7a40f315f63bc62d9f8183222c6fdf199a59cf27fb23a67d6b002: Status 404 returned error can't find the container with id 17d337144bd7a40f315f63bc62d9f8183222c6fdf199a59cf27fb23a67d6b002 Sep 18 03:20:26.107851 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:26.107815 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5fqzd" event={"ID":"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1","Type":"ContainerStarted","Data":"17d337144bd7a40f315f63bc62d9f8183222c6fdf199a59cf27fb23a67d6b002"} Sep 18 03:20:27.115385 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:27.115356 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5fqzd" event={"ID":"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1","Type":"ContainerStarted","Data":"2c77f8f5c9ad44f2dc47358c515fa4e0a446599ec96513e7fc3b6340f532ff0b"} Sep 18 03:20:27.115695 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:27.115435 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5fqzd" event={"ID":"bcc9eec9-e533-4cd1-b3db-bed11c4b76c1","Type":"ContainerStarted","Data":"6369bfc0b505bbab8a64d29077dae92bf1efadfe7d97d8be7c2c63381377c508"} Sep 18 03:20:27.115695 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:27.115500 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5fqzd" Sep 18 03:20:27.154643 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:27.154591 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5fqzd" podStartSLOduration=129.423161929 podStartE2EDuration="2m11.154573981s" totalImagesPullingTime="1.731412052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:20:27.153851964 +0000 UTC m=+166.369293139" watchObservedRunningTime="2026-09-18 03:20:27.154573981 +0000 UTC m=+166.370015155" Sep 18 03:20:29.502007 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:29.501980 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:29.592316 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:29.592279 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:30.290549 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:30.290521 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:32.349671 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:32.349635 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:32.476383 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:32.476297 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:33.142498 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:33.142464 2580 generic.go:356] "Generic (PLEG): container finished" podID="ff2a6591-33f5-4c87-99cf-969fa772ff61" containerID="cb0b61f06a8718484f98de1d2d22d906be4188555912582243279f481f251fb5" exitCode=0 Sep 18 03:20:33.142688 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:33.142538 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" event={"ID":"ff2a6591-33f5-4c87-99cf-969fa772ff61","Type":"ContainerDied","Data":"cb0b61f06a8718484f98de1d2d22d906be4188555912582243279f481f251fb5"} Sep 18 03:20:33.142858 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:33.142845 2580 scope.go:117] "RemoveContainer" containerID="cb0b61f06a8718484f98de1d2d22d906be4188555912582243279f481f251fb5" Sep 18 03:20:34.069909 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:34.069879 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5fqzd_bcc9eec9-e533-4cd1-b3db-bed11c4b76c1/dns/0.log" Sep 18 03:20:34.143832 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:34.143804 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5fqzd_bcc9eec9-e533-4cd1-b3db-bed11c4b76c1/kube-rbac-proxy/0.log" Sep 18 03:20:34.148363 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:34.148335 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-vkr82" event={"ID":"ff2a6591-33f5-4c87-99cf-969fa772ff61","Type":"ContainerStarted","Data":"f13672f0e04722fe248595de645f145a6dc75e62a4bbc1a6b9aab8c0d8bff9d9"} Sep 18 03:20:34.275166 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:34.275139 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d29f9_b2814e15-58e8-4144-becf-e6af9314b295/dns-node-resolver/0.log" Sep 18 03:20:36.617198 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:36.617164 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:36.693403 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:36.693369 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:37.121466 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:37.121442 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5fqzd" Sep 18 03:20:41.637297 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:41.637266 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:41.744456 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:41.744428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:42.181216 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:42.181155 2580 generic.go:356] "Generic (PLEG): container finished" podID="ac414f76-9056-4e31-a0d9-e09ffc8451a4" containerID="31e627e00b5e29cb466865de6cac782d05ae8f91746d61d284710617f42b0ce1" exitCode=0 Sep 18 03:20:42.181313 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:42.181220 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" event={"ID":"ac414f76-9056-4e31-a0d9-e09ffc8451a4","Type":"ContainerDied","Data":"31e627e00b5e29cb466865de6cac782d05ae8f91746d61d284710617f42b0ce1"} Sep 18 03:20:42.181491 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:42.181479 2580 scope.go:117] "RemoveContainer" containerID="31e627e00b5e29cb466865de6cac782d05ae8f91746d61d284710617f42b0ce1" Sep 18 03:20:43.192883 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:43.192846 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-kwwd9" event={"ID":"ac414f76-9056-4e31-a0d9-e09ffc8451a4","Type":"ContainerStarted","Data":"ee98a73d3c5372081c264ee5cd9b2df014eb8bcd4c338c993ea58df8f916a305"} Sep 18 03:20:50.588752 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:50.588718 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:20:50.702218 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:20:50.702178 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:03.404086 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:03.404034 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:03.420449 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:03.420423 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:04.296979 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:04.296952 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:12.926782 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:12.926747 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:13.027368 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:13.027332 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:17.602873 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:17.602840 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 18 03:21:17.603313 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:17.603174 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="alertmanager" containerID="cri-o://256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706" gracePeriod=120 Sep 18 03:21:17.603313 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:17.603246 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy-web" containerID="cri-o://dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677" gracePeriod=120 Sep 18 03:21:17.603313 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:17.603277 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="config-reloader" containerID="cri-o://178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901" gracePeriod=120 Sep 18 03:21:17.603471 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:17.603306 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="prom-label-proxy" containerID="cri-o://4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931" gracePeriod=120 Sep 18 03:21:17.603471 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:17.603246 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy-metric" containerID="cri-o://52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27" gracePeriod=120 Sep 18 03:21:17.603471 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:17.603316 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy" containerID="cri-o://0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650" gracePeriod=120 Sep 18 03:21:18.339403 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.339370 2580 generic.go:356] "Generic (PLEG): container finished" podID="f53fd542-85b5-466c-96b7-558b737918b2" containerID="4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931" exitCode=0 Sep 18 03:21:18.339403 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.339395 2580 generic.go:356] "Generic (PLEG): container finished" podID="f53fd542-85b5-466c-96b7-558b737918b2" containerID="0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650" exitCode=0 Sep 18 03:21:18.339403 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.339401 2580 generic.go:356] "Generic (PLEG): container finished" podID="f53fd542-85b5-466c-96b7-558b737918b2" containerID="178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901" exitCode=0 Sep 18 03:21:18.339403 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.339407 2580 generic.go:356] "Generic (PLEG): container finished" podID="f53fd542-85b5-466c-96b7-558b737918b2" containerID="256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706" exitCode=0 Sep 18 03:21:18.339668 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.339437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerDied","Data":"4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931"} Sep 18 03:21:18.339668 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.339495 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerDied","Data":"0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650"} Sep 18 03:21:18.339668 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.339520 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerDied","Data":"178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901"} Sep 18 03:21:18.339668 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.339544 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerDied","Data":"256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706"} Sep 18 03:21:18.849123 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.849097 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:21:18.931037 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.930949 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931037 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.930991 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-tls-assets\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931037 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931019 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-main-db\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931042 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-cluster-tls-config\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931091 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r9qk\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-kube-api-access-6r9qk\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931114 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-config-out\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931145 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-web-config\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931188 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-main-tls\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931223 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-trusted-ca-bundle\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931274 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931674 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931335 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931674 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931361 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-config-volume\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931674 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931392 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 03:21:18.931674 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931422 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-metrics-client-ca\") pod \"f53fd542-85b5-466c-96b7-558b737918b2\" (UID: \"f53fd542-85b5-466c-96b7-558b737918b2\") " Sep 18 03:21:18.931875 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931790 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:21:18.931875 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.931789 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-main-db\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:18.932553 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.932522 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:21:18.934375 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.934345 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:18.935760 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.935149 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-kube-api-access-6r9qk" (OuterVolumeSpecName: "kube-api-access-6r9qk") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "kube-api-access-6r9qk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:21:18.935760 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.935287 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-config-volume" (OuterVolumeSpecName: "config-volume") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:18.935760 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.935745 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:21:18.935971 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.935753 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:18.936238 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.936214 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-config-out" (OuterVolumeSpecName: "config-out") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 03:21:18.938144 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.938028 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:18.938144 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.938119 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:18.942433 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.942404 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:18.949881 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:18.949860 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-web-config" (OuterVolumeSpecName: "web-config") pod "f53fd542-85b5-466c-96b7-558b737918b2" (UID: "f53fd542-85b5-466c-96b7-558b737918b2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:19.032656 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032625 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-web-config\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032656 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032652 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-main-tls\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032666 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032675 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032685 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032693 2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-config-volume\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032701 2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f53fd542-85b5-466c-96b7-558b737918b2-metrics-client-ca\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032709 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032717 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-tls-assets\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032726 2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f53fd542-85b5-466c-96b7-558b737918b2-cluster-tls-config\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032735 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r9qk\" (UniqueName: \"kubernetes.io/projected/f53fd542-85b5-466c-96b7-558b737918b2-kube-api-access-6r9qk\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.032822 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.032743 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f53fd542-85b5-466c-96b7-558b737918b2-config-out\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:19.346079 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.346026 2580 generic.go:356] "Generic (PLEG): container finished" podID="f53fd542-85b5-466c-96b7-558b737918b2" containerID="52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27" exitCode=0 Sep 18 03:21:19.346212 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.346072 2580 generic.go:356] "Generic (PLEG): container finished" podID="f53fd542-85b5-466c-96b7-558b737918b2" containerID="dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677" exitCode=0 Sep 18 03:21:19.346212 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.346107 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerDied","Data":"52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27"} Sep 18 03:21:19.346212 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.346176 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerDied","Data":"dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677"} Sep 18 03:21:19.346212 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.346119 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 18 03:21:19.346212 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.346203 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f53fd542-85b5-466c-96b7-558b737918b2","Type":"ContainerDied","Data":"0bb63f02c4ddb8e46f021f4ffbb861483bdf250e2df8c546660a5abc4d6299f0"} Sep 18 03:21:19.346384 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.346232 2580 scope.go:117] "RemoveContainer" containerID="4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931" Sep 18 03:21:19.353870 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.353854 2580 scope.go:117] "RemoveContainer" containerID="52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27" Sep 18 03:21:19.361626 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.361606 2580 scope.go:117] "RemoveContainer" containerID="0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650" Sep 18 03:21:19.368684 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.368658 2580 scope.go:117] "RemoveContainer" containerID="dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677" Sep 18 03:21:19.375789 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.375741 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 18 03:21:19.376341 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.376320 2580 scope.go:117] "RemoveContainer" containerID="178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901" Sep 18 03:21:19.383997 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.383896 2580 scope.go:117] "RemoveContainer" containerID="256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706" Sep 18 03:21:19.386664 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.386641 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 18 03:21:19.391477 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.391450 2580 scope.go:117] "RemoveContainer" containerID="f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7" Sep 18 03:21:19.400128 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.400102 2580 scope.go:117] "RemoveContainer" containerID="4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931" Sep 18 03:21:19.400432 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:19.400411 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931\": container with ID starting with 4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931 not found: ID does not exist" containerID="4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931" Sep 18 03:21:19.400493 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.400445 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931"} err="failed to get container status \"4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931\": rpc error: code = NotFound desc = could not find container \"4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931\": container with ID starting with 4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931 not found: ID does not exist" Sep 18 03:21:19.400493 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.400476 2580 scope.go:117] "RemoveContainer" containerID="52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27" Sep 18 03:21:19.400736 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:19.400718 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27\": container with ID starting with 52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27 not found: ID does not exist" containerID="52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27" Sep 18 03:21:19.400797 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.400745 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27"} err="failed to get container status \"52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27\": rpc error: code = NotFound desc = could not find container \"52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27\": container with ID starting with 52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27 not found: ID does not exist" Sep 18 03:21:19.400797 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.400768 2580 scope.go:117] "RemoveContainer" containerID="0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650" Sep 18 03:21:19.401031 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:19.401012 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650\": container with ID starting with 0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650 not found: ID does not exist" containerID="0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650" Sep 18 03:21:19.401097 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.401037 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650"} err="failed to get container status \"0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650\": rpc error: code = NotFound desc = could not find container \"0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650\": container with ID starting with 0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650 not found: ID does not exist" Sep 18 03:21:19.401097 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.401072 2580 scope.go:117] "RemoveContainer" containerID="dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677" Sep 18 03:21:19.401328 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:19.401308 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677\": container with ID starting with dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677 not found: ID does not exist" containerID="dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677" Sep 18 03:21:19.401392 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.401337 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677"} err="failed to get container status \"dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677\": rpc error: code = NotFound desc = could not find container \"dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677\": container with ID starting with dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677 not found: ID does not exist" Sep 18 03:21:19.401392 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.401356 2580 scope.go:117] "RemoveContainer" containerID="178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901" Sep 18 03:21:19.401633 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:19.401618 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901\": container with ID starting with 178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901 not found: ID does not exist" containerID="178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901" Sep 18 03:21:19.401689 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.401638 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901"} err="failed to get container status \"178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901\": rpc error: code = NotFound desc = could not find container \"178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901\": container with ID starting with 178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901 not found: ID does not exist" Sep 18 03:21:19.401689 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.401667 2580 scope.go:117] "RemoveContainer" containerID="256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706" Sep 18 03:21:19.401875 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:19.401858 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706\": container with ID starting with 256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706 not found: ID does not exist" containerID="256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706" Sep 18 03:21:19.401914 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.401880 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706"} err="failed to get container status \"256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706\": rpc error: code = NotFound desc = could not find container \"256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706\": container with ID starting with 256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706 not found: ID does not exist" Sep 18 03:21:19.401914 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.401895 2580 scope.go:117] "RemoveContainer" containerID="f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7" Sep 18 03:21:19.402167 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:19.402147 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7\": container with ID starting with f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7 not found: ID does not exist" containerID="f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7" Sep 18 03:21:19.402257 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.402168 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7"} err="failed to get container status \"f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7\": rpc error: code = NotFound desc = could not find container \"f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7\": container with ID starting with f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7 not found: ID does not exist" Sep 18 03:21:19.402257 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.402186 2580 scope.go:117] "RemoveContainer" containerID="4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931" Sep 18 03:21:19.402409 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.402392 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931"} err="failed to get container status \"4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931\": rpc error: code = NotFound desc = could not find container \"4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931\": container with ID starting with 4248b81a00c36d633a16c057a14d5618dfd8a4180d0b8acee765157feb048931 not found: ID does not exist" Sep 18 03:21:19.402454 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.402410 2580 scope.go:117] "RemoveContainer" containerID="52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27" Sep 18 03:21:19.402619 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.402604 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27"} err="failed to get container status \"52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27\": rpc error: code = NotFound desc = could not find container \"52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27\": container with ID starting with 52ad65555c958d85fc2f4b9e8dac50818ac7190da1db3f656351007965571e27 not found: ID does not exist" Sep 18 03:21:19.402619 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.402618 2580 scope.go:117] "RemoveContainer" containerID="0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650" Sep 18 03:21:19.402823 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.402808 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650"} err="failed to get container status \"0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650\": rpc error: code = NotFound desc = could not find container \"0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650\": container with ID starting with 0caaf26cb8998f64bc93b163c5e3c26f0e2fec5922a7c3d969eb5edc70e4b650 not found: ID does not exist" Sep 18 03:21:19.402909 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.402823 2580 scope.go:117] "RemoveContainer" containerID="dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677" Sep 18 03:21:19.403024 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.403005 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677"} err="failed to get container status \"dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677\": rpc error: code = NotFound desc = could not find container \"dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677\": container with ID starting with dccac21328cda06393dc1fc73c33f345263f004b1344ff1754ca06023ed42677 not found: ID does not exist" Sep 18 03:21:19.403121 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.403026 2580 scope.go:117] "RemoveContainer" containerID="178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901" Sep 18 03:21:19.403317 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.403300 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901"} err="failed to get container status \"178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901\": rpc error: code = NotFound desc = could not find container \"178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901\": container with ID starting with 178fd9b336dc3c261b74d80592a882f5d6a7c8cdb505c4066965629f27a9c901 not found: ID does not exist" Sep 18 03:21:19.403371 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.403317 2580 scope.go:117] "RemoveContainer" containerID="256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706" Sep 18 03:21:19.403533 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.403516 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706"} err="failed to get container status \"256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706\": rpc error: code = NotFound desc = could not find container \"256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706\": container with ID starting with 256d2115c2c6cd9baa02414eba74f82375f34087f0eaa3a63c7ef79e24624706 not found: ID does not exist" Sep 18 03:21:19.403588 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.403533 2580 scope.go:117] "RemoveContainer" containerID="f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7" Sep 18 03:21:19.403758 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:19.403742 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7"} err="failed to get container status \"f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7\": rpc error: code = NotFound desc = could not find container \"f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7\": container with ID starting with f6603619c634725dce31ee66ae2ea173abd2fb0a52ff8d3dde69d8169a6848f7 not found: ID does not exist" Sep 18 03:21:21.298801 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:21.298765 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f53fd542-85b5-466c-96b7-558b737918b2" path="/var/lib/kubelet/pods/f53fd542-85b5-466c-96b7-558b737918b2/volumes" Sep 18 03:21:21.905631 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:21.905597 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:21:21.905908 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:21.905870 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="prometheus" containerID="cri-o://01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" gracePeriod=600 Sep 18 03:21:21.905908 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:21.905891 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy" containerID="cri-o://219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" gracePeriod=600 Sep 18 03:21:21.906136 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:21.905913 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="thanos-sidecar" containerID="cri-o://27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" gracePeriod=600 Sep 18 03:21:21.906136 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:21.905939 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy-web" containerID="cri-o://cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" gracePeriod=600 Sep 18 03:21:21.906136 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:21.905962 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="config-reloader" containerID="cri-o://ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" gracePeriod=600 Sep 18 03:21:21.906136 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:21.905963 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" gracePeriod=600 Sep 18 03:21:22.150097 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.150070 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.260942 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.260867 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-web-config\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.260942 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.260905 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-0\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.260942 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.260940 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-metrics-client-ca\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.261232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.260964 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-config\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.261232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.260988 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-tls\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.261232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261022 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.261232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261045 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-1\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.261232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261087 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-thanos-prometheus-http-client-file\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.261232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261122 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-trusted-ca-bundle\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.261232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261157 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-kube-rbac-proxy\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.261232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261201 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-db\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.261232 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261224 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261269 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-tls-assets\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261315 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-kubelet-serving-ca-bundle\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261338 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvtc6\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-kube-api-access-fvtc6\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261375 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-metrics-client-certs\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261398 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-2\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261422 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-grpc-tls\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261450 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-serving-certs-ca-bundle\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.261478 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-config-out\") pod \"65b538f8-2611-4e7b-a389-e52e771368e1\" (UID: \"65b538f8-2611-4e7b-a389-e52e771368e1\") " Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.262440 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.262556 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.262924 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 03:21:22.263734 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.262946 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:21:22.264477 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.263910 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:21:22.264477 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.263953 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:21:22.264477 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.264398 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:21:22.264477 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.264464 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 03:21:22.265590 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.265555 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-config-out" (OuterVolumeSpecName: "config-out") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 03:21:22.266575 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.266540 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:22.266801 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.266754 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:22.266801 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.266774 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-config" (OuterVolumeSpecName: "config") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:22.267846 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.267812 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:22.268045 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.268024 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:21:22.268045 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.268041 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:22.268225 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.268170 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:22.268760 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.268728 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:22.268760 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.268749 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:22.269021 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.268998 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-kube-api-access-fvtc6" (OuterVolumeSpecName: "kube-api-access-fvtc6") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "kube-api-access-fvtc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:21:22.276662 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.276641 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-web-config" (OuterVolumeSpecName: "web-config") pod "65b538f8-2611-4e7b-a389-e52e771368e1" (UID: "65b538f8-2611-4e7b-a389-e52e771368e1"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 03:21:22.276758 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.276741 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:22.361695 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361657 2580 generic.go:356] "Generic (PLEG): container finished" podID="65b538f8-2611-4e7b-a389-e52e771368e1" containerID="7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" exitCode=0 Sep 18 03:21:22.361695 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361683 2580 generic.go:356] "Generic (PLEG): container finished" podID="65b538f8-2611-4e7b-a389-e52e771368e1" containerID="219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" exitCode=0 Sep 18 03:21:22.361695 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361692 2580 generic.go:356] "Generic (PLEG): container finished" podID="65b538f8-2611-4e7b-a389-e52e771368e1" containerID="cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" exitCode=0 Sep 18 03:21:22.361695 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361698 2580 generic.go:356] "Generic (PLEG): container finished" podID="65b538f8-2611-4e7b-a389-e52e771368e1" containerID="27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" exitCode=0 Sep 18 03:21:22.361695 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361705 2580 generic.go:356] "Generic (PLEG): container finished" podID="65b538f8-2611-4e7b-a389-e52e771368e1" containerID="ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" exitCode=0 Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361712 2580 generic.go:356] "Generic (PLEG): container finished" podID="65b538f8-2611-4e7b-a389-e52e771368e1" containerID="01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" exitCode=0 Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361730 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361732 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerDied","Data":"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef"} Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361799 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerDied","Data":"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3"} Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerDied","Data":"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31"} Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361851 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerDied","Data":"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477"} Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361871 2580 scope.go:117] "RemoveContainer" containerID="7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361875 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerDied","Data":"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88"} Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361966 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerDied","Data":"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19"} Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.361993 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65b538f8-2611-4e7b-a389-e52e771368e1","Type":"ContainerDied","Data":"34a5caf940f5a1779350d04dabef7db68fd92fc732939b9f45fade58454248f6"} Sep 18 03:21:22.362354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362345 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-tls-assets\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362365 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362380 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvtc6\" (UniqueName: \"kubernetes.io/projected/65b538f8-2611-4e7b-a389-e52e771368e1-kube-api-access-fvtc6\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362397 2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-metrics-client-certs\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362410 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362424 2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-grpc-tls\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362437 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362450 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-config-out\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362463 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-web-config\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362475 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362489 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-configmap-metrics-client-ca\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362502 2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-config\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362515 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362529 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362544 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362559 2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362573 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362586 2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-kube-rbac-proxy\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362601 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/65b538f8-2611-4e7b-a389-e52e771368e1-prometheus-k8s-db\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.362826 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.362615 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/65b538f8-2611-4e7b-a389-e52e771368e1-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:21:22.370623 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.370602 2580 scope.go:117] "RemoveContainer" containerID="219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" Sep 18 03:21:22.377543 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.377524 2580 scope.go:117] "RemoveContainer" containerID="cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" Sep 18 03:21:22.384342 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.384325 2580 scope.go:117] "RemoveContainer" containerID="27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" Sep 18 03:21:22.390947 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.390928 2580 scope.go:117] "RemoveContainer" containerID="ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" Sep 18 03:21:22.397829 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.397812 2580 scope.go:117] "RemoveContainer" containerID="01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" Sep 18 03:21:22.405873 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.405843 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:21:22.406422 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.406400 2580 scope.go:117] "RemoveContainer" containerID="f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3" Sep 18 03:21:22.414297 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.414282 2580 scope.go:117] "RemoveContainer" containerID="7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" Sep 18 03:21:22.414569 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:22.414552 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": container with ID starting with 7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef not found: ID does not exist" containerID="7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" Sep 18 03:21:22.414638 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.414576 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef"} err="failed to get container status \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": rpc error: code = NotFound desc = could not find container \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": container with ID starting with 7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef not found: ID does not exist" Sep 18 03:21:22.414638 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.414595 2580 scope.go:117] "RemoveContainer" containerID="219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" Sep 18 03:21:22.414877 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.414855 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:21:22.414912 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:22.414867 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": container with ID starting with 219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3 not found: ID does not exist" containerID="219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" Sep 18 03:21:22.414912 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.414892 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3"} err="failed to get container status \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": rpc error: code = NotFound desc = could not find container \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": container with ID starting with 219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3 not found: ID does not exist" Sep 18 03:21:22.414983 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.414913 2580 scope.go:117] "RemoveContainer" containerID="cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" Sep 18 03:21:22.415197 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:22.415181 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": container with ID starting with cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31 not found: ID does not exist" containerID="cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" Sep 18 03:21:22.415256 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.415199 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31"} err="failed to get container status \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": rpc error: code = NotFound desc = could not find container \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": container with ID starting with cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31 not found: ID does not exist" Sep 18 03:21:22.415256 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.415212 2580 scope.go:117] "RemoveContainer" containerID="27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" Sep 18 03:21:22.415430 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:22.415415 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": container with ID starting with 27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477 not found: ID does not exist" containerID="27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" Sep 18 03:21:22.415473 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.415435 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477"} err="failed to get container status \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": rpc error: code = NotFound desc = could not find container \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": container with ID starting with 27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477 not found: ID does not exist" Sep 18 03:21:22.415473 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.415450 2580 scope.go:117] "RemoveContainer" containerID="ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" Sep 18 03:21:22.415686 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:22.415672 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": container with ID starting with ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88 not found: ID does not exist" containerID="ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" Sep 18 03:21:22.415733 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.415688 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88"} err="failed to get container status \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": rpc error: code = NotFound desc = could not find container \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": container with ID starting with ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88 not found: ID does not exist" Sep 18 03:21:22.415733 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.415699 2580 scope.go:117] "RemoveContainer" containerID="01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" Sep 18 03:21:22.415894 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:22.415877 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": container with ID starting with 01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19 not found: ID does not exist" containerID="01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" Sep 18 03:21:22.415940 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.415897 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19"} err="failed to get container status \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": rpc error: code = NotFound desc = could not find container \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": container with ID starting with 01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19 not found: ID does not exist" Sep 18 03:21:22.415940 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.415907 2580 scope.go:117] "RemoveContainer" containerID="f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3" Sep 18 03:21:22.416223 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:21:22.416206 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": container with ID starting with f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3 not found: ID does not exist" containerID="f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3" Sep 18 03:21:22.416289 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.416225 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3"} err="failed to get container status \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": rpc error: code = NotFound desc = could not find container \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": container with ID starting with f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3 not found: ID does not exist" Sep 18 03:21:22.416289 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.416237 2580 scope.go:117] "RemoveContainer" containerID="7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" Sep 18 03:21:22.416456 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.416436 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef"} err="failed to get container status \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": rpc error: code = NotFound desc = could not find container \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": container with ID starting with 7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef not found: ID does not exist" Sep 18 03:21:22.416500 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.416457 2580 scope.go:117] "RemoveContainer" containerID="219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" Sep 18 03:21:22.416668 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.416651 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3"} err="failed to get container status \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": rpc error: code = NotFound desc = could not find container \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": container with ID starting with 219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3 not found: ID does not exist" Sep 18 03:21:22.416668 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.416667 2580 scope.go:117] "RemoveContainer" containerID="cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" Sep 18 03:21:22.416874 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.416859 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31"} err="failed to get container status \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": rpc error: code = NotFound desc = could not find container \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": container with ID starting with cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31 not found: ID does not exist" Sep 18 03:21:22.416934 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.416874 2580 scope.go:117] "RemoveContainer" containerID="27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" Sep 18 03:21:22.417117 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.417101 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477"} err="failed to get container status \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": rpc error: code = NotFound desc = could not find container \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": container with ID starting with 27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477 not found: ID does not exist" Sep 18 03:21:22.417117 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.417116 2580 scope.go:117] "RemoveContainer" containerID="ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" Sep 18 03:21:22.417382 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.417364 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88"} err="failed to get container status \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": rpc error: code = NotFound desc = could not find container \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": container with ID starting with ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88 not found: ID does not exist" Sep 18 03:21:22.417428 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.417385 2580 scope.go:117] "RemoveContainer" containerID="01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" Sep 18 03:21:22.417630 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.417610 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19"} err="failed to get container status \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": rpc error: code = NotFound desc = could not find container \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": container with ID starting with 01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19 not found: ID does not exist" Sep 18 03:21:22.417688 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.417630 2580 scope.go:117] "RemoveContainer" containerID="f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3" Sep 18 03:21:22.417863 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.417848 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3"} err="failed to get container status \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": rpc error: code = NotFound desc = could not find container \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": container with ID starting with f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3 not found: ID does not exist" Sep 18 03:21:22.417905 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.417864 2580 scope.go:117] "RemoveContainer" containerID="7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" Sep 18 03:21:22.418105 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.418089 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef"} err="failed to get container status \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": rpc error: code = NotFound desc = could not find container \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": container with ID starting with 7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef not found: ID does not exist" Sep 18 03:21:22.418161 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.418106 2580 scope.go:117] "RemoveContainer" containerID="219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" Sep 18 03:21:22.418337 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.418320 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3"} err="failed to get container status \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": rpc error: code = NotFound desc = could not find container \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": container with ID starting with 219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3 not found: ID does not exist" Sep 18 03:21:22.418375 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.418338 2580 scope.go:117] "RemoveContainer" containerID="cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" Sep 18 03:21:22.418559 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.418543 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31"} err="failed to get container status \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": rpc error: code = NotFound desc = could not find container \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": container with ID starting with cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31 not found: ID does not exist" Sep 18 03:21:22.418559 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.418560 2580 scope.go:117] "RemoveContainer" containerID="27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" Sep 18 03:21:22.418806 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.418787 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477"} err="failed to get container status \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": rpc error: code = NotFound desc = could not find container \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": container with ID starting with 27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477 not found: ID does not exist" Sep 18 03:21:22.418856 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.418807 2580 scope.go:117] "RemoveContainer" containerID="ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" Sep 18 03:21:22.419038 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419023 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88"} err="failed to get container status \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": rpc error: code = NotFound desc = could not find container \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": container with ID starting with ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88 not found: ID does not exist" Sep 18 03:21:22.419107 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419038 2580 scope.go:117] "RemoveContainer" containerID="01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" Sep 18 03:21:22.419283 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419263 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19"} err="failed to get container status \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": rpc error: code = NotFound desc = could not find container \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": container with ID starting with 01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19 not found: ID does not exist" Sep 18 03:21:22.419354 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419285 2580 scope.go:117] "RemoveContainer" containerID="f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3" Sep 18 03:21:22.419521 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419505 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3"} err="failed to get container status \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": rpc error: code = NotFound desc = could not find container \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": container with ID starting with f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3 not found: ID does not exist" Sep 18 03:21:22.419566 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419521 2580 scope.go:117] "RemoveContainer" containerID="7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" Sep 18 03:21:22.419725 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419708 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef"} err="failed to get container status \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": rpc error: code = NotFound desc = could not find container \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": container with ID starting with 7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef not found: ID does not exist" Sep 18 03:21:22.419785 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419727 2580 scope.go:117] "RemoveContainer" containerID="219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" Sep 18 03:21:22.419977 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419959 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3"} err="failed to get container status \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": rpc error: code = NotFound desc = could not find container \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": container with ID starting with 219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3 not found: ID does not exist" Sep 18 03:21:22.420016 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.419977 2580 scope.go:117] "RemoveContainer" containerID="cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" Sep 18 03:21:22.420234 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.420212 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31"} err="failed to get container status \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": rpc error: code = NotFound desc = could not find container \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": container with ID starting with cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31 not found: ID does not exist" Sep 18 03:21:22.420234 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.420233 2580 scope.go:117] "RemoveContainer" containerID="27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" Sep 18 03:21:22.420474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.420456 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477"} err="failed to get container status \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": rpc error: code = NotFound desc = could not find container \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": container with ID starting with 27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477 not found: ID does not exist" Sep 18 03:21:22.420524 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.420474 2580 scope.go:117] "RemoveContainer" containerID="ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" Sep 18 03:21:22.420679 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.420663 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88"} err="failed to get container status \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": rpc error: code = NotFound desc = could not find container \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": container with ID starting with ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88 not found: ID does not exist" Sep 18 03:21:22.420721 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.420679 2580 scope.go:117] "RemoveContainer" containerID="01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" Sep 18 03:21:22.420869 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.420847 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19"} err="failed to get container status \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": rpc error: code = NotFound desc = could not find container \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": container with ID starting with 01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19 not found: ID does not exist" Sep 18 03:21:22.420913 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.420869 2580 scope.go:117] "RemoveContainer" containerID="f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3" Sep 18 03:21:22.421065 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421036 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3"} err="failed to get container status \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": rpc error: code = NotFound desc = could not find container \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": container with ID starting with f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3 not found: ID does not exist" Sep 18 03:21:22.421106 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421078 2580 scope.go:117] "RemoveContainer" containerID="7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" Sep 18 03:21:22.421263 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421249 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef"} err="failed to get container status \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": rpc error: code = NotFound desc = could not find container \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": container with ID starting with 7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef not found: ID does not exist" Sep 18 03:21:22.421305 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421262 2580 scope.go:117] "RemoveContainer" containerID="219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" Sep 18 03:21:22.421481 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421466 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3"} err="failed to get container status \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": rpc error: code = NotFound desc = could not find container \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": container with ID starting with 219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3 not found: ID does not exist" Sep 18 03:21:22.421548 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421481 2580 scope.go:117] "RemoveContainer" containerID="cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" Sep 18 03:21:22.421693 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421673 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31"} err="failed to get container status \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": rpc error: code = NotFound desc = could not find container \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": container with ID starting with cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31 not found: ID does not exist" Sep 18 03:21:22.421735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421694 2580 scope.go:117] "RemoveContainer" containerID="27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" Sep 18 03:21:22.421937 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421906 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477"} err="failed to get container status \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": rpc error: code = NotFound desc = could not find container \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": container with ID starting with 27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477 not found: ID does not exist" Sep 18 03:21:22.421937 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.421928 2580 scope.go:117] "RemoveContainer" containerID="ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" Sep 18 03:21:22.422140 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422124 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88"} err="failed to get container status \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": rpc error: code = NotFound desc = could not find container \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": container with ID starting with ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88 not found: ID does not exist" Sep 18 03:21:22.422187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422140 2580 scope.go:117] "RemoveContainer" containerID="01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" Sep 18 03:21:22.422338 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422319 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19"} err="failed to get container status \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": rpc error: code = NotFound desc = could not find container \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": container with ID starting with 01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19 not found: ID does not exist" Sep 18 03:21:22.422384 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422341 2580 scope.go:117] "RemoveContainer" containerID="f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3" Sep 18 03:21:22.422535 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422519 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3"} err="failed to get container status \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": rpc error: code = NotFound desc = could not find container \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": container with ID starting with f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3 not found: ID does not exist" Sep 18 03:21:22.422580 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422535 2580 scope.go:117] "RemoveContainer" containerID="7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef" Sep 18 03:21:22.422720 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422704 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef"} err="failed to get container status \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": rpc error: code = NotFound desc = could not find container \"7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef\": container with ID starting with 7a29153ce1fa314e92e4a1d5be0371b01cd0f14854e437b59985bcd801a729ef not found: ID does not exist" Sep 18 03:21:22.422764 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422722 2580 scope.go:117] "RemoveContainer" containerID="219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3" Sep 18 03:21:22.422914 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422899 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3"} err="failed to get container status \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": rpc error: code = NotFound desc = could not find container \"219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3\": container with ID starting with 219710b1574d80611a0d8809745868cf4475c5c13d64bdaba46a5dd48f727ef3 not found: ID does not exist" Sep 18 03:21:22.422956 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.422916 2580 scope.go:117] "RemoveContainer" containerID="cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31" Sep 18 03:21:22.423135 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.423119 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31"} err="failed to get container status \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": rpc error: code = NotFound desc = could not find container \"cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31\": container with ID starting with cb2ccaf8e3c3efe360ef1953c28a7d6f4d61e46acbf31cee63f004f003527a31 not found: ID does not exist" Sep 18 03:21:22.423187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.423136 2580 scope.go:117] "RemoveContainer" containerID="27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477" Sep 18 03:21:22.423315 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.423301 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477"} err="failed to get container status \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": rpc error: code = NotFound desc = could not find container \"27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477\": container with ID starting with 27e2580c9ac0a4f2ae029c978076a04ae7a6c5c0a979e942a89f469b33f90477 not found: ID does not exist" Sep 18 03:21:22.423357 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.423316 2580 scope.go:117] "RemoveContainer" containerID="ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88" Sep 18 03:21:22.423527 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.423510 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88"} err="failed to get container status \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": rpc error: code = NotFound desc = could not find container \"ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88\": container with ID starting with ff308a1e673fdcccc83a7dd81c8f0ca3ff05a9b2abe0d58136907f802ed76c88 not found: ID does not exist" Sep 18 03:21:22.423569 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.423527 2580 scope.go:117] "RemoveContainer" containerID="01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19" Sep 18 03:21:22.423751 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.423735 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19"} err="failed to get container status \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": rpc error: code = NotFound desc = could not find container \"01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19\": container with ID starting with 01817d05c0fa5973301262d3ef1c557b3775073b12beeb792d27649f947bad19 not found: ID does not exist" Sep 18 03:21:22.423796 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.423752 2580 scope.go:117] "RemoveContainer" containerID="f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3" Sep 18 03:21:22.423955 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.423929 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3"} err="failed to get container status \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": rpc error: code = NotFound desc = could not find container \"f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3\": container with ID starting with f89437074fe0d89165a76806f016ec467858430f03a4d4907d2ed5f6f39e7cc3 not found: ID does not exist" Sep 18 03:21:22.430986 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.430968 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:22.527984 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.527919 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:21:22.528217 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528205 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="config-reloader" Sep 18 03:21:22.528264 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528218 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="config-reloader" Sep 18 03:21:22.528264 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528228 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy" Sep 18 03:21:22.528264 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528233 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy" Sep 18 03:21:22.528264 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528242 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy-web" Sep 18 03:21:22.528264 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528248 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy-web" Sep 18 03:21:22.528264 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528255 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="config-reloader" Sep 18 03:21:22.528264 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528260 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="config-reloader" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528267 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="prom-label-proxy" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528272 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="prom-label-proxy" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528277 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="init-config-reloader" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528282 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="init-config-reloader" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528289 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="init-config-reloader" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528294 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="init-config-reloader" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528299 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy-metric" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528304 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy-metric" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528309 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy-web" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528315 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy-web" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528323 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="prometheus" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528327 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="prometheus" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528333 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy-thanos" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528338 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy-thanos" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528354 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="alertmanager" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528361 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="alertmanager" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528369 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="thanos-sidecar" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528374 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="thanos-sidecar" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528381 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528385 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528423 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="prom-label-proxy" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528431 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="thanos-sidecar" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528438 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528443 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy-web" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528448 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy-thanos" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528453 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="alertmanager" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528458 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="kube-rbac-proxy" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528463 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy-metric" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528468 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="config-reloader" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528475 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53fd542-85b5-466c-96b7-558b737918b2" containerName="kube-rbac-proxy-web" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528480 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="config-reloader" Sep 18 03:21:22.528474 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.528486 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" containerName="prometheus" Sep 18 03:21:22.533565 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.533548 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.541465 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.541445 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 18 03:21:22.541673 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.541655 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 18 03:21:22.541756 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.541728 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 18 03:21:22.541841 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.541808 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pnn5x\"" Sep 18 03:21:22.543222 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.543206 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 18 03:21:22.543298 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.543272 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 18 03:21:22.543390 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.543373 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 18 03:21:22.543461 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.543449 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 18 03:21:22.543541 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.543525 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 18 03:21:22.548152 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.548136 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 18 03:21:22.548669 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.548657 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-coagstnrr4qo8\"" Sep 18 03:21:22.549095 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.549083 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 18 03:21:22.555555 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.555540 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 18 03:21:22.555947 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.555925 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 18 03:21:22.556120 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.556104 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 18 03:21:22.564813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.564791 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 18 03:21:22.564813 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.564804 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 18 03:21:22.569255 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.569235 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:21:22.665705 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.665705 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.665932 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.665932 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.665932 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.665932 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665867 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.665932 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665894 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.665932 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665920 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665945 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.665974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666089 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7pzx\" (UniqueName: \"kubernetes.io/projected/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-kube-api-access-q7pzx\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666149 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-config\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666187 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666487 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.666487 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.666228 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.766937 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.766896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7pzx\" (UniqueName: \"kubernetes.io/projected/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-kube-api-access-q7pzx\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.766937 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.766940 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767190 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767190 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767190 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-config\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767344 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767197 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767344 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767344 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767344 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767344 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767344 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767341 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767682 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767682 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767399 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767682 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767682 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767682 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767682 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767534 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767682 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767559 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767682 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.767682 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.767646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.768391 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.768311 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.768903 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.768875 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.769512 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.769482 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.770293 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.770268 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.770397 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.770295 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.770397 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.770314 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-config\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.770637 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.770613 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.771542 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.771244 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.771542 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.771489 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.771692 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.771587 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.771942 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.771710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.771942 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.771789 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.772239 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.772217 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.772457 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.772437 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.772901 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.772874 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.773242 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.773220 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.773449 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.773430 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.773825 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.773810 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.774245 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.774217 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.789468 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.789424 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7pzx\" (UniqueName: \"kubernetes.io/projected/1ab12bf1-efdb-4be5-b0df-41482eff5c4d-kube-api-access-q7pzx\") pod \"prometheus-k8s-0\" (UID: \"1ab12bf1-efdb-4be5-b0df-41482eff5c4d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:22.842599 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:22.842572 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:23.146277 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:23.146255 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 18 03:21:23.148124 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:21:23.148097 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ab12bf1_efdb_4be5_b0df_41482eff5c4d.slice/crio-248e838d7c05818cdfd61c7bae384413a2ffc9c178e5fcce764a484b38417fb3 WatchSource:0}: Error finding container 248e838d7c05818cdfd61c7bae384413a2ffc9c178e5fcce764a484b38417fb3: Status 404 returned error can't find the container with id 248e838d7c05818cdfd61c7bae384413a2ffc9c178e5fcce764a484b38417fb3 Sep 18 03:21:23.297406 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:23.297379 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65b538f8-2611-4e7b-a389-e52e771368e1" path="/var/lib/kubelet/pods/65b538f8-2611-4e7b-a389-e52e771368e1/volumes" Sep 18 03:21:23.367428 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:23.367396 2580 generic.go:356] "Generic (PLEG): container finished" podID="1ab12bf1-efdb-4be5-b0df-41482eff5c4d" containerID="8665519523d6a071dc1f0f19095ba6e065cd1facfdfb4b8fe45d481835e5c537" exitCode=0 Sep 18 03:21:23.367791 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:23.367481 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ab12bf1-efdb-4be5-b0df-41482eff5c4d","Type":"ContainerDied","Data":"8665519523d6a071dc1f0f19095ba6e065cd1facfdfb4b8fe45d481835e5c537"} Sep 18 03:21:23.367791 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:23.367539 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ab12bf1-efdb-4be5-b0df-41482eff5c4d","Type":"ContainerStarted","Data":"248e838d7c05818cdfd61c7bae384413a2ffc9c178e5fcce764a484b38417fb3"} Sep 18 03:21:24.375199 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:24.375165 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ab12bf1-efdb-4be5-b0df-41482eff5c4d","Type":"ContainerStarted","Data":"022b9cc27e9f8ceecaff24335bc5d9847e3aa3ce114483746c7d14612e9157fb"} Sep 18 03:21:24.375544 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:24.375228 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ab12bf1-efdb-4be5-b0df-41482eff5c4d","Type":"ContainerStarted","Data":"a6a588e17c64f7b47e18df7da4ff8743dc1c669ca90c4594ca418d24509ace51"} Sep 18 03:21:24.375544 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:24.375254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ab12bf1-efdb-4be5-b0df-41482eff5c4d","Type":"ContainerStarted","Data":"f0d97755b2bf8f4eef1f05c3161435803474c70edb877cfee956a30e227c57ec"} Sep 18 03:21:24.375544 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:24.375278 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ab12bf1-efdb-4be5-b0df-41482eff5c4d","Type":"ContainerStarted","Data":"b3c3ffbe63ed2e0ec52cbae24fdcba41f8b898c20808d941ea9bd0d8a35367c5"} Sep 18 03:21:24.375544 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:24.375305 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ab12bf1-efdb-4be5-b0df-41482eff5c4d","Type":"ContainerStarted","Data":"2016f66a4504592ef937eee83595c874a32865042e74703900f7be78e547d523"} Sep 18 03:21:24.375544 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:24.375328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ab12bf1-efdb-4be5-b0df-41482eff5c4d","Type":"ContainerStarted","Data":"b68d12d4e14fc39b16b0c2b9b67882e12f17cb34682074f6abc549c3b1631d09"} Sep 18 03:21:24.425771 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:24.425727 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.406212384 podStartE2EDuration="2.425710926s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.498542ms" isStatefulPod=false podCreationTimestamp="2026-09-18 03:21:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:21:24.424740926 +0000 UTC m=+223.640182101" watchObservedRunningTime="2026-09-18 03:21:24.425710926 +0000 UTC m=+223.641152101" Sep 18 03:21:24.952839 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:24.952801 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:25.060252 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:25.060212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:27.843226 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:27.843177 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:21:31.452031 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:31.451997 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:33.202947 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:33.202914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:33.486724 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:33.486644 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:33.603488 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:33.603457 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:36.638621 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:36.638586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:36.755973 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:36.755942 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:21:36.872320 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:21:36.872286 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:22:03.807198 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:03.807124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:22:03.890863 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:03.890830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:22:22.363233 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:22.363200 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:22:22.843214 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:22.843178 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:22:22.859242 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:22.859219 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:22:23.618328 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:23.618288 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 18 03:22:29.036235 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:29.036204 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:22:29.147077 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:29.147021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:22:41.191945 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:41.191915 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/2.log" Sep 18 03:22:41.201829 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:41.201810 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/2.log" Sep 18 03:22:41.203197 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:41.203177 2580 kubelet.go:1648] "Image garbage collection succeeded" Sep 18 03:22:44.091149 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:44.091117 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:22:44.202850 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:44.202799 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:22:57.959456 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:57.959420 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:22:58.143697 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:22:58.143663 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:01.997461 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:01.997425 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:02.228211 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:02.228179 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:03.502965 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:03.502929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:03.680656 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:03.680624 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:05.823517 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:05.823481 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:06.190319 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:06.190240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:07.678321 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:07.678288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:07.794735 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:07.794699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:13.172757 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:13.172716 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:13.343220 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:13.343170 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:14.102725 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:14.102691 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:29.085454 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:29.085422 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:29.197449 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:29.197414 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:39.627429 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:39.627364 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:40.786250 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:40.786192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:44.583172 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:44.583135 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:49.816802 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:49.816768 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:50.315520 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:50.315487 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:50.466264 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:50.466233 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:23:52.677883 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:23:52.677837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:24:34.187740 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:24:34.187701 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:24:45.057443 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:24:45.057410 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:24:45.198705 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:24:45.198660 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:24:45.562395 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:24:45.562365 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:24:45.955569 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:24:45.955497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:17.486514 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:17.486437 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:17.580068 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:17.580024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:17.683263 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:17.683232 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:18.054290 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:18.054255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:18.155757 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:18.155729 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:23.425394 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:23.425343 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:23.505600 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:23.505575 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:28.288790 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:28.288739 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:28.389754 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:28.389720 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:28.465710 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:28.465665 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:33.343911 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:33.343862 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:33.445630 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:33.445594 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:36.245809 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:36.245775 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:36.643715 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:36.643681 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:36.718478 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:36.718445 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:39.641370 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:39.641336 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:39.725670 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:39.725634 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:57.549538 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.549502 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:25:57.622343 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.622309 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-zn9fl"] Sep 18 03:25:57.625798 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.625782 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" Sep 18 03:25:57.635183 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.635164 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dnf4k\"" Sep 18 03:25:57.636623 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.636608 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 18 03:25:57.636715 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.636625 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 18 03:25:57.647610 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.647587 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-zn9fl"] Sep 18 03:25:57.773760 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.773732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwmb9\" (UniqueName: \"kubernetes.io/projected/c77c2517-895a-4386-a219-1522075b02ee-kube-api-access-nwmb9\") pod \"test-trainjob-workflow-node-0-0-zn9fl\" (UID: \"c77c2517-895a-4386-a219-1522075b02ee\") " pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" Sep 18 03:25:57.874421 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.874394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwmb9\" (UniqueName: \"kubernetes.io/projected/c77c2517-895a-4386-a219-1522075b02ee-kube-api-access-nwmb9\") pod \"test-trainjob-workflow-node-0-0-zn9fl\" (UID: \"c77c2517-895a-4386-a219-1522075b02ee\") " pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" Sep 18 03:25:57.884285 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.884252 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwmb9\" (UniqueName: \"kubernetes.io/projected/c77c2517-895a-4386-a219-1522075b02ee-kube-api-access-nwmb9\") pod \"test-trainjob-workflow-node-0-0-zn9fl\" (UID: \"c77c2517-895a-4386-a219-1522075b02ee\") " pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" Sep 18 03:25:57.942841 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:57.942807 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" Sep 18 03:25:58.218240 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:58.218163 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-zn9fl"] Sep 18 03:25:58.221155 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:25:58.221125 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc77c2517_895a_4386_a219_1522075b02ee.slice/crio-0c5b1cf94652851f93d670083630204c71bbde3f2d3851e5e2ede736613e7891 WatchSource:0}: Error finding container 0c5b1cf94652851f93d670083630204c71bbde3f2d3851e5e2ede736613e7891: Status 404 returned error can't find the container with id 0c5b1cf94652851f93d670083630204c71bbde3f2d3851e5e2ede736613e7891 Sep 18 03:25:58.223316 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:58.223295 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 03:25:58.413070 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:25:58.413015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" event={"ID":"c77c2517-895a-4386-a219-1522075b02ee","Type":"ContainerStarted","Data":"0c5b1cf94652851f93d670083630204c71bbde3f2d3851e5e2ede736613e7891"} Sep 18 03:26:32.123597 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:26:32.123551 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:27:04.753452 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:04.753409 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" event={"ID":"c77c2517-895a-4386-a219-1522075b02ee","Type":"ContainerStarted","Data":"74d3198c50efd95f8ee9cece0d01f92f2a6ce164af16546eaf7e6fe50b66cb29"} Sep 18 03:27:04.793364 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:04.793314 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" podStartSLOduration=1.8096518910000001 podStartE2EDuration="1m7.793295479s" totalImagesPullingTime="1m5.983643588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:27:04.791845853 +0000 UTC m=+564.007287059" watchObservedRunningTime="2026-09-18 03:27:04.793295479 +0000 UTC m=+564.008736656" Sep 18 03:27:06.763689 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:06.763627 2580 generic.go:356] "Generic (PLEG): container finished" podID="c77c2517-895a-4386-a219-1522075b02ee" containerID="74d3198c50efd95f8ee9cece0d01f92f2a6ce164af16546eaf7e6fe50b66cb29" exitCode=0 Sep 18 03:27:06.764114 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:06.763700 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" event={"ID":"c77c2517-895a-4386-a219-1522075b02ee","Type":"ContainerDied","Data":"74d3198c50efd95f8ee9cece0d01f92f2a6ce164af16546eaf7e6fe50b66cb29"} Sep 18 03:27:08.063733 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:08.063702 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" Sep 18 03:27:08.110692 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:08.110670 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwmb9\" (UniqueName: \"kubernetes.io/projected/c77c2517-895a-4386-a219-1522075b02ee-kube-api-access-nwmb9\") pod \"c77c2517-895a-4386-a219-1522075b02ee\" (UID: \"c77c2517-895a-4386-a219-1522075b02ee\") " Sep 18 03:27:08.112829 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:08.112805 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c77c2517-895a-4386-a219-1522075b02ee-kube-api-access-nwmb9" (OuterVolumeSpecName: "kube-api-access-nwmb9") pod "c77c2517-895a-4386-a219-1522075b02ee" (UID: "c77c2517-895a-4386-a219-1522075b02ee"). InnerVolumeSpecName "kube-api-access-nwmb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:27:08.211911 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:08.211885 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwmb9\" (UniqueName: \"kubernetes.io/projected/c77c2517-895a-4386-a219-1522075b02ee-kube-api-access-nwmb9\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:27:08.773537 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:08.773511 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" Sep 18 03:27:08.773684 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:08.773564 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-trainjob-workflow-node-0-0-zn9fl" event={"ID":"c77c2517-895a-4386-a219-1522075b02ee","Type":"ContainerDied","Data":"0c5b1cf94652851f93d670083630204c71bbde3f2d3851e5e2ede736613e7891"} Sep 18 03:27:08.773684 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:08.773607 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c5b1cf94652851f93d670083630204c71bbde3f2d3851e5e2ede736613e7891" Sep 18 03:27:09.900514 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:09.900482 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-zn9fl"] Sep 18 03:27:09.911096 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:09.911071 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-zn9fl"] Sep 18 03:27:09.959966 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:09.959945 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:27:10.256231 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:10.256162 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:27:10.366369 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:10.366348 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:27:10.711726 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:10.711699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:27:10.815017 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:10.814985 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:27:11.298700 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.298672 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c77c2517-895a-4386-a219-1522075b02ee" path="/var/lib/kubelet/pods/c77c2517-895a-4386-a219-1522075b02ee/volumes" Sep 18 03:27:11.632440 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.632415 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gvpbc/must-gather-2r6hj"] Sep 18 03:27:11.632701 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.632689 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c77c2517-895a-4386-a219-1522075b02ee" containerName="node" Sep 18 03:27:11.632741 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.632702 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c77c2517-895a-4386-a219-1522075b02ee" containerName="node" Sep 18 03:27:11.632786 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.632777 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c77c2517-895a-4386-a219-1522075b02ee" containerName="node" Sep 18 03:27:11.661146 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.661121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:27:11.710203 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.710180 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gvpbc/must-gather-2r6hj"] Sep 18 03:27:11.710300 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.710261 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:27:11.715626 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.715608 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gvpbc\"/\"default-dockercfg-g4nmb\"" Sep 18 03:27:11.715775 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.715758 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gvpbc\"/\"kube-root-ca.crt\"" Sep 18 03:27:11.716719 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.716698 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gvpbc\"/\"openshift-service-ca.crt\"" Sep 18 03:27:11.765567 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.765535 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:27:11.840680 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.840659 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5rfr\" (UniqueName: \"kubernetes.io/projected/1e4412a8-d0c1-4d10-a016-6f512754270c-kube-api-access-z5rfr\") pod \"must-gather-2r6hj\" (UID: \"1e4412a8-d0c1-4d10-a016-6f512754270c\") " pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:27:11.840775 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.840702 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e4412a8-d0c1-4d10-a016-6f512754270c-must-gather-output\") pod \"must-gather-2r6hj\" (UID: \"1e4412a8-d0c1-4d10-a016-6f512754270c\") " pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:27:11.941141 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.941091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5rfr\" (UniqueName: \"kubernetes.io/projected/1e4412a8-d0c1-4d10-a016-6f512754270c-kube-api-access-z5rfr\") pod \"must-gather-2r6hj\" (UID: \"1e4412a8-d0c1-4d10-a016-6f512754270c\") " pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:27:11.941141 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.941134 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e4412a8-d0c1-4d10-a016-6f512754270c-must-gather-output\") pod \"must-gather-2r6hj\" (UID: \"1e4412a8-d0c1-4d10-a016-6f512754270c\") " pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:27:11.941439 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.941423 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e4412a8-d0c1-4d10-a016-6f512754270c-must-gather-output\") pod \"must-gather-2r6hj\" (UID: \"1e4412a8-d0c1-4d10-a016-6f512754270c\") " pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:27:11.950194 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:11.950173 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5rfr\" (UniqueName: \"kubernetes.io/projected/1e4412a8-d0c1-4d10-a016-6f512754270c-kube-api-access-z5rfr\") pod \"must-gather-2r6hj\" (UID: \"1e4412a8-d0c1-4d10-a016-6f512754270c\") " pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:27:12.031281 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:12.031259 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:27:12.325850 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:12.325820 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gvpbc/must-gather-2r6hj"] Sep 18 03:27:12.328577 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:27:12.328552 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e4412a8_d0c1_4d10_a016_6f512754270c.slice/crio-00bcce2f9f95fe1bae1c3102faaa627f93519fd6ba8edf461cf204011b24d171 WatchSource:0}: Error finding container 00bcce2f9f95fe1bae1c3102faaa627f93519fd6ba8edf461cf204011b24d171: Status 404 returned error can't find the container with id 00bcce2f9f95fe1bae1c3102faaa627f93519fd6ba8edf461cf204011b24d171 Sep 18 03:27:12.790125 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:12.790017 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" event={"ID":"1e4412a8-d0c1-4d10-a016-6f512754270c","Type":"ContainerStarted","Data":"00bcce2f9f95fe1bae1c3102faaa627f93519fd6ba8edf461cf204011b24d171"} Sep 18 03:27:20.826664 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:20.826623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" event={"ID":"1e4412a8-d0c1-4d10-a016-6f512754270c","Type":"ContainerStarted","Data":"62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78"} Sep 18 03:27:20.826664 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:20.826673 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" event={"ID":"1e4412a8-d0c1-4d10-a016-6f512754270c","Type":"ContainerStarted","Data":"b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191"} Sep 18 03:27:20.845506 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:20.845443 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" podStartSLOduration=2.291164101 podStartE2EDuration="9.845425181s" totalImagesPullingTime="7.55426108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 03:27:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:27:20.843537335 +0000 UTC m=+580.058978510" watchObservedRunningTime="2026-09-18 03:27:20.845425181 +0000 UTC m=+580.060866358" Sep 18 03:27:41.223699 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:41.223671 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/2.log" Sep 18 03:27:41.230256 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:27:41.230238 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/2.log" Sep 18 03:28:13.056326 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:13.056290 2580 generic.go:356] "Generic (PLEG): container finished" podID="1e4412a8-d0c1-4d10-a016-6f512754270c" containerID="b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191" exitCode=0 Sep 18 03:28:13.056713 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:13.056365 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" event={"ID":"1e4412a8-d0c1-4d10-a016-6f512754270c","Type":"ContainerDied","Data":"b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191"} Sep 18 03:28:13.056753 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:13.056713 2580 scope.go:117] "RemoveContainer" containerID="b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191" Sep 18 03:28:13.360213 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:13.360186 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gvpbc_must-gather-2r6hj_1e4412a8-d0c1-4d10-a016-6f512754270c/gather/0.log" Sep 18 03:28:13.738866 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:13.738785 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:28:13.889832 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:13.889798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:28:13.989901 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:13.989825 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:28:16.583219 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:16.583196 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fzpz8_994a76dc-2129-4b42-8bb2-a51be4c091d7/global-pull-secret-syncer/0.log" Sep 18 03:28:16.770520 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:16.770491 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" Sep 18 03:28:16.854901 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:16.854832 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-90.ec2.internal_c5d8fca0c5700e570250443bdef4a41a/haproxy/0.log" Sep 18 03:28:18.710121 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:18.710085 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-gvpbc/must-gather-2r6hj"] Sep 18 03:28:18.710546 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:18.710312 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" containerName="copy" containerID="cri-o://62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78" gracePeriod=2 Sep 18 03:28:18.712457 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:18.712422 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-gvpbc/must-gather-2r6hj"] Sep 18 03:28:18.712835 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:28:18.712810 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-2r6hj\" is forbidden: User \"system:node:ip-10-0-129-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gvpbc\": no relationship found between node 'ip-10-0-129-90.ec2.internal' and this object" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:28:18.943325 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:18.943299 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gvpbc_must-gather-2r6hj_1e4412a8-d0c1-4d10-a016-6f512754270c/copy/0.log" Sep 18 03:28:18.945926 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:18.945906 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:28:18.948384 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:28:18.948360 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-2r6hj\" is forbidden: User \"system:node:ip-10-0-129-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gvpbc\": no relationship found between node 'ip-10-0-129-90.ec2.internal' and this object" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:28:19.031997 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.031920 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5rfr\" (UniqueName: \"kubernetes.io/projected/1e4412a8-d0c1-4d10-a016-6f512754270c-kube-api-access-z5rfr\") pod \"1e4412a8-d0c1-4d10-a016-6f512754270c\" (UID: \"1e4412a8-d0c1-4d10-a016-6f512754270c\") " Sep 18 03:28:19.032170 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.032000 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e4412a8-d0c1-4d10-a016-6f512754270c-must-gather-output\") pod \"1e4412a8-d0c1-4d10-a016-6f512754270c\" (UID: \"1e4412a8-d0c1-4d10-a016-6f512754270c\") " Sep 18 03:28:19.033276 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.033253 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e4412a8-d0c1-4d10-a016-6f512754270c-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "1e4412a8-d0c1-4d10-a016-6f512754270c" (UID: "1e4412a8-d0c1-4d10-a016-6f512754270c"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 03:28:19.034325 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.034302 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e4412a8-d0c1-4d10-a016-6f512754270c-kube-api-access-z5rfr" (OuterVolumeSpecName: "kube-api-access-z5rfr") pod "1e4412a8-d0c1-4d10-a016-6f512754270c" (UID: "1e4412a8-d0c1-4d10-a016-6f512754270c"). InnerVolumeSpecName "kube-api-access-z5rfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 03:28:19.081372 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.081350 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gvpbc_must-gather-2r6hj_1e4412a8-d0c1-4d10-a016-6f512754270c/copy/0.log" Sep 18 03:28:19.081729 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.081705 2580 generic.go:356] "Generic (PLEG): container finished" podID="1e4412a8-d0c1-4d10-a016-6f512754270c" containerID="62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78" exitCode=143 Sep 18 03:28:19.081810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.081758 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:28:19.081810 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.081777 2580 scope.go:117] "RemoveContainer" containerID="62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78" Sep 18 03:28:19.084729 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:28:19.084695 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-2r6hj\" is forbidden: User \"system:node:ip-10-0-129-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gvpbc\": no relationship found between node 'ip-10-0-129-90.ec2.internal' and this object" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:28:19.090128 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.090111 2580 scope.go:117] "RemoveContainer" containerID="b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191" Sep 18 03:28:19.092598 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:28:19.092570 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-2r6hj\" is forbidden: User \"system:node:ip-10-0-129-90.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gvpbc\": no relationship found between node 'ip-10-0-129-90.ec2.internal' and this object" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" pod="openshift-must-gather-gvpbc/must-gather-2r6hj" Sep 18 03:28:19.100912 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.100899 2580 scope.go:117] "RemoveContainer" containerID="62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78" Sep 18 03:28:19.101198 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:28:19.101179 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78\": container with ID starting with 62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78 not found: ID does not exist" containerID="62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78" Sep 18 03:28:19.101254 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.101204 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78"} err="failed to get container status \"62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78\": rpc error: code = NotFound desc = could not find container \"62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78\": container with ID starting with 62e34ebbfcb392a301d8384616cc78f0f25fe6c25333f0f909b27a23c1d25c78 not found: ID does not exist" Sep 18 03:28:19.101254 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.101219 2580 scope.go:117] "RemoveContainer" containerID="b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191" Sep 18 03:28:19.101460 ip-10-0-129-90 kubenswrapper[2580]: E0918 03:28:19.101443 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191\": container with ID starting with b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191 not found: ID does not exist" containerID="b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191" Sep 18 03:28:19.101509 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.101471 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191"} err="failed to get container status \"b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191\": rpc error: code = NotFound desc = could not find container \"b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191\": container with ID starting with b3d528613ce0da5b01fe16bab6b42068c136f988724717ec5b557b29aca75191 not found: ID does not exist" Sep 18 03:28:19.132666 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.132647 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5rfr\" (UniqueName: \"kubernetes.io/projected/1e4412a8-d0c1-4d10-a016-6f512754270c-kube-api-access-z5rfr\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:28:19.132666 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.132667 2580 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e4412a8-d0c1-4d10-a016-6f512754270c-must-gather-output\") on node \"ip-10-0-129-90.ec2.internal\" DevicePath \"\"" Sep 18 03:28:19.298069 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.297991 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" path="/var/lib/kubelet/pods/1e4412a8-d0c1-4d10-a016-6f512754270c/volumes" Sep 18 03:28:19.500628 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.500595 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:28:19.561137 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:19.561106 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b8sbn_b3b737b9-7cab-470a-8c89-d20c07544375/konnectivity-agent/0.log" limit=1024 Sep 18 03:28:20.113618 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.113583 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-lp7d7_a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2/kube-state-metrics/0.log" Sep 18 03:28:20.169369 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.169328 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-lp7d7_a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2/kube-rbac-proxy-main/0.log" Sep 18 03:28:20.210290 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.210263 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-lp7d7_a9b0ae5d-3fbd-414b-9cfa-235a9276f6b2/kube-rbac-proxy-self/0.log" Sep 18 03:28:20.460284 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.460250 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fhfrj_b86d4755-ee15-4b98-87b9-589f06a77272/node-exporter/0.log" Sep 18 03:28:20.482384 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.482363 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fhfrj_b86d4755-ee15-4b98-87b9-589f06a77272/kube-rbac-proxy/0.log" Sep 18 03:28:20.507032 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.507015 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fhfrj_b86d4755-ee15-4b98-87b9-589f06a77272/init-textfile/0.log" Sep 18 03:28:20.770171 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.770097 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ab12bf1-efdb-4be5-b0df-41482eff5c4d/prometheus/0.log" Sep 18 03:28:20.814168 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.814143 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ab12bf1-efdb-4be5-b0df-41482eff5c4d/config-reloader/0.log" Sep 18 03:28:20.871426 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.871409 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ab12bf1-efdb-4be5-b0df-41482eff5c4d/thanos-sidecar/0.log" Sep 18 03:28:20.898146 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.898126 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ab12bf1-efdb-4be5-b0df-41482eff5c4d/kube-rbac-proxy-web/0.log" Sep 18 03:28:20.928538 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.928519 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ab12bf1-efdb-4be5-b0df-41482eff5c4d/kube-rbac-proxy/0.log" Sep 18 03:28:20.954030 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.953994 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ab12bf1-efdb-4be5-b0df-41482eff5c4d/kube-rbac-proxy-thanos/0.log" Sep 18 03:28:20.983669 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:20.983626 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1ab12bf1-efdb-4be5-b0df-41482eff5c4d/init-config-reloader/0.log" Sep 18 03:28:21.191587 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:21.191568 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58cdb4884b-m92cq_944d25f1-d00f-4e5f-864b-0621a45388ef/thanos-query/0.log" Sep 18 03:28:21.217093 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:21.217072 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58cdb4884b-m92cq_944d25f1-d00f-4e5f-864b-0621a45388ef/kube-rbac-proxy-web/0.log" Sep 18 03:28:21.243277 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:21.243258 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58cdb4884b-m92cq_944d25f1-d00f-4e5f-864b-0621a45388ef/kube-rbac-proxy/0.log" Sep 18 03:28:21.270046 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:21.270025 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58cdb4884b-m92cq_944d25f1-d00f-4e5f-864b-0621a45388ef/prom-label-proxy/0.log" Sep 18 03:28:21.292280 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:21.292261 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58cdb4884b-m92cq_944d25f1-d00f-4e5f-864b-0621a45388ef/kube-rbac-proxy-rules/0.log" Sep 18 03:28:21.318750 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:21.318732 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-58cdb4884b-m92cq_944d25f1-d00f-4e5f-864b-0621a45388ef/kube-rbac-proxy-metrics/0.log" Sep 18 03:28:23.126601 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.126572 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/2.log" Sep 18 03:28:23.130896 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.130874 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-2j7sn_d79bd980-6c95-4cf7-8f81-e53a08cc190e/console-operator/3.log" Sep 18 03:28:23.859717 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.859683 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv"] Sep 18 03:28:23.860074 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.860046 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" containerName="gather" Sep 18 03:28:23.860128 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.860077 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" containerName="gather" Sep 18 03:28:23.860128 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.860089 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" containerName="copy" Sep 18 03:28:23.860128 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.860094 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" containerName="copy" Sep 18 03:28:23.860225 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.860147 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" containerName="gather" Sep 18 03:28:23.860225 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.860160 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e4412a8-d0c1-4d10-a016-6f512754270c" containerName="copy" Sep 18 03:28:23.865127 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.865110 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:23.868436 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.868414 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nwskp\"/\"openshift-service-ca.crt\"" Sep 18 03:28:23.868545 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.868439 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nwskp\"/\"kube-root-ca.crt\"" Sep 18 03:28:23.869887 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.869870 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nwskp\"/\"default-dockercfg-zxqsd\"" Sep 18 03:28:23.872485 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.872454 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv"] Sep 18 03:28:23.971532 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.971501 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-podres\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:23.971691 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.971552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-sys\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:23.971691 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.971569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-lib-modules\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:23.971691 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.971654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xrm8\" (UniqueName: \"kubernetes.io/projected/f5f75046-f999-4077-937d-7c1ce6a9f72f-kube-api-access-2xrm8\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:23.971819 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:23.971693 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-proc\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.072893 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.072863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-podres\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.073042 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.072910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-sys\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.073042 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.072925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-lib-modules\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.073042 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.072950 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xrm8\" (UniqueName: \"kubernetes.io/projected/f5f75046-f999-4077-937d-7c1ce6a9f72f-kube-api-access-2xrm8\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.073042 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.072978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-proc\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.073042 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.073009 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-sys\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.073269 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.073043 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-podres\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.073269 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.073076 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-proc\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.073269 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.073093 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5f75046-f999-4077-937d-7c1ce6a9f72f-lib-modules\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.082034 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.082007 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xrm8\" (UniqueName: \"kubernetes.io/projected/f5f75046-f999-4077-937d-7c1ce6a9f72f-kube-api-access-2xrm8\") pod \"perf-node-gather-daemonset-2fcwv\" (UID: \"f5f75046-f999-4077-937d-7c1ce6a9f72f\") " pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.175967 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.175905 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:24.447665 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:24.447591 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv"] Sep 18 03:28:24.450563 ip-10-0-129-90 kubenswrapper[2580]: W0918 03:28:24.450534 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf5f75046_f999_4077_937d_7c1ce6a9f72f.slice/crio-03612b6d02367d16964da3e62e2af9b298be2d389c2d3786e6e4d398306b66c0 WatchSource:0}: Error finding container 03612b6d02367d16964da3e62e2af9b298be2d389c2d3786e6e4d398306b66c0: Status 404 returned error can't find the container with id 03612b6d02367d16964da3e62e2af9b298be2d389c2d3786e6e4d398306b66c0 Sep 18 03:28:25.069341 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:25.069314 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5fqzd_bcc9eec9-e533-4cd1-b3db-bed11c4b76c1/dns/0.log" Sep 18 03:28:25.104126 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:25.104097 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5fqzd_bcc9eec9-e533-4cd1-b3db-bed11c4b76c1/kube-rbac-proxy/0.log" Sep 18 03:28:25.109345 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:25.109318 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" event={"ID":"f5f75046-f999-4077-937d-7c1ce6a9f72f","Type":"ContainerStarted","Data":"8cc5078e16ac2af599527ed7f08290c951ecf14fe6acf8a98c585233ac236ed4"} Sep 18 03:28:25.109470 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:25.109365 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" event={"ID":"f5f75046-f999-4077-937d-7c1ce6a9f72f","Type":"ContainerStarted","Data":"03612b6d02367d16964da3e62e2af9b298be2d389c2d3786e6e4d398306b66c0"} Sep 18 03:28:25.109470 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:25.109429 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:25.129089 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:25.129029 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" podStartSLOduration=2.129015737 podStartE2EDuration="2.129015737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 03:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 03:28:25.127702791 +0000 UTC m=+644.343143967" watchObservedRunningTime="2026-09-18 03:28:25.129015737 +0000 UTC m=+644.344456911" Sep 18 03:28:25.217928 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:25.217906 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d29f9_b2814e15-58e8-4144-becf-e6af9314b295/dns-node-resolver/0.log" Sep 18 03:28:25.677786 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:25.677761 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-94655745f-w4r6z_b994e472-f9f7-405f-9fa8-d441eb56646b/registry/0.log" Sep 18 03:28:25.734489 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:25.734464 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fdnrl_5b477d64-4f58-4f2b-aca6-f898589820cb/node-ca/0.log" Sep 18 03:28:26.616960 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:26.616927 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c9b9b47d6-s4lnj_be3bf17c-370d-414e-891d-bb24a5768021/router/0.log" Sep 18 03:28:26.975984 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:26.975905 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-826hv_991c83a8-e37c-4043-b60a-13c67bcca116/serve-healthcheck-canary/0.log" Sep 18 03:28:27.409389 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:27.409356 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-vkr82_ff2a6591-33f5-4c87-99cf-969fa772ff61/insights-operator/0.log" Sep 18 03:28:27.409550 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:27.409540 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-vkr82_ff2a6591-33f5-4c87-99cf-969fa772ff61/insights-operator/1.log" Sep 18 03:28:27.435721 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:27.435698 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2hm5z_0cf9a91b-ff80-4653-9046-1fbcc82cd395/kube-rbac-proxy/0.log" Sep 18 03:28:27.461039 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:27.461019 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2hm5z_0cf9a91b-ff80-4653-9046-1fbcc82cd395/exporter/0.log" Sep 18 03:28:27.486172 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:27.486148 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2hm5z_0cf9a91b-ff80-4653-9046-1fbcc82cd395/extractor/0.log" Sep 18 03:28:31.123275 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:31.123246 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nwskp/perf-node-gather-daemonset-2fcwv" Sep 18 03:28:33.450804 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:33.450775 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-kwwd9_ac414f76-9056-4e31-a0d9-e09ffc8451a4/kube-storage-version-migrator-operator/1.log" Sep 18 03:28:33.451705 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:33.451686 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-kwwd9_ac414f76-9056-4e31-a0d9-e09ffc8451a4/kube-storage-version-migrator-operator/0.log" Sep 18 03:28:34.780791 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:34.780756 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvlk7_7af14471-f55a-469b-a673-9f9903c399e3/kube-multus-additional-cni-plugins/0.log" Sep 18 03:28:34.806404 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:34.806378 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvlk7_7af14471-f55a-469b-a673-9f9903c399e3/egress-router-binary-copy/0.log" Sep 18 03:28:34.831095 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:34.831075 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvlk7_7af14471-f55a-469b-a673-9f9903c399e3/cni-plugins/0.log" Sep 18 03:28:34.854183 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:34.854164 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvlk7_7af14471-f55a-469b-a673-9f9903c399e3/bond-cni-plugin/0.log" Sep 18 03:28:34.878960 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:34.878941 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvlk7_7af14471-f55a-469b-a673-9f9903c399e3/routeoverride-cni/0.log" Sep 18 03:28:34.902980 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:34.902962 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvlk7_7af14471-f55a-469b-a673-9f9903c399e3/whereabouts-cni-bincopy/0.log" Sep 18 03:28:34.926223 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:34.926168 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvlk7_7af14471-f55a-469b-a673-9f9903c399e3/whereabouts-cni/0.log" Sep 18 03:28:35.015506 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:35.015475 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fldjl_48c50c68-f633-4e98-8823-587153372c93/kube-multus/0.log" Sep 18 03:28:35.085032 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:35.085006 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ssqd2_ffd6a923-32d3-4212-8231-20b173a6a6a7/network-metrics-daemon/0.log" Sep 18 03:28:35.106316 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:35.106294 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ssqd2_ffd6a923-32d3-4212-8231-20b173a6a6a7/kube-rbac-proxy/0.log" Sep 18 03:28:36.662696 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:36.662666 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m7g_9c20f405-c42c-4c5b-b789-465af48ce967/ovn-controller/0.log" Sep 18 03:28:36.690602 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:36.690578 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m7g_9c20f405-c42c-4c5b-b789-465af48ce967/ovn-acl-logging/0.log" Sep 18 03:28:36.722779 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:36.722722 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m7g_9c20f405-c42c-4c5b-b789-465af48ce967/kube-rbac-proxy-node/0.log" Sep 18 03:28:36.747884 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:36.747864 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m7g_9c20f405-c42c-4c5b-b789-465af48ce967/kube-rbac-proxy-ovn-metrics/0.log" Sep 18 03:28:36.775805 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:36.775787 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m7g_9c20f405-c42c-4c5b-b789-465af48ce967/northd/0.log" Sep 18 03:28:36.798323 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:36.798305 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m7g_9c20f405-c42c-4c5b-b789-465af48ce967/nbdb/0.log" Sep 18 03:28:36.822858 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:36.822834 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m7g_9c20f405-c42c-4c5b-b789-465af48ce967/sbdb/0.log" Sep 18 03:28:36.862739 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:36.862719 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m7g_9c20f405-c42c-4c5b-b789-465af48ce967/ovnkube-controller/0.log" Sep 18 03:28:37.865611 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:37.865574 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7cbc45fddc-rz2qd_b53834df-51de-4d06-bc0e-24f87f33af0c/check-endpoints/0.log" Sep 18 03:28:37.916877 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:37.916853 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-hwg2m_f274a119-a4cd-4900-9ebc-4c3d7c023542/network-check-target-container/0.log" Sep 18 03:28:38.888753 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:38.888722 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-t2nmd_a1091cd1-b35a-49b0-b97c-a15ee4efbebe/iptables-alerter/0.log" Sep 18 03:28:39.576538 ip-10-0-129-90 kubenswrapper[2580]: I0918 03:28:39.576504 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-pbf5f_dddf8f39-e67b-48a2-bac4-2399e3718e57/tuned/0.log"